We tested the following as an experiment, but not extensively because we decided it wasn't worth it. Still, it worked, so I offer it for people who would think that this could be fun enough to work into the regular flow.
Imagine how it might affect verbal components of spellcasting. I imagine that losing precision when invoking the magic words would possibly cause the spell to fail, or be cast as something else (unintended consequences). Try Disadvantage.
All that coughing and sniffling can't possibly be good for stealth. You can somewhat control it, but when you get down to attempting something you are more likely to fail --— Disadvantage.
If a Perception check involves the sense of smell (exe.g., the wafting odorsodor of a goblin squad wafting down a corridor), then you can also try Disadvantage.
You could also reduce the number of hit points and spells recovered when resting, sort of like how you don't feel up to par when you wake up with a cold.