The general rule is that damage is a roll
As you have noted, a critical permits an extra roll of damage dice, but does not change any modifications to the roll. That might make it seem like 'flat damage' is not augmented by a critical, but note that damage is defined as a die or dice. From the PHB on Damage Rolls (emphasis mine):
Each weapon, spell, and harmful monster ability specifies the damage it deals. You roll the damage die or dice, add any modifiers, and apply the damage to your target. Magic weapons, special abilities, and other factors can grant a bonus to damage.
Damage requires a roll of a damage die, or dice, by definition. Modifiers are qualified as 'any', indicating that they may or may not be present, but no such qualification is made for damage dice. Damage is always a roll; we are not told 'You take any flat damage value, roll and add any damage die or dice, add any modifiers, and then apply the total damage to your target'.
Unarmed strikes are an exception
The PHB section on melee attacks says:
Instead of using a weapon to make a melee weapon attack, you can use an unarmed strike: a punch, kick, head-butt, or similar forceful blow (none of which count as weapons). On a hit, an unarmed strike deals bludgeoning damage equal to 1 + your Strength modifier. You are proficient with your unarmed strikes.
The specific rule about unarmed strikes overrides the general rule that damage is rolled. Because there are no damage dice to roll for an unarmed strike, a critical hit has no dice to double. A critical hit on an unexceptional unarmed attack does no extra damage.