No, fatigue does not generally effect spellcasting
As you've correctly noted, neither the fatigued or exhausted conditions specify any penalty which applies to most attempts to cast spells, and there is no mention of fatigue affecting spellcasting in the concentration rules. The most significant indirect effect is that the strength/dexterity penalty will hamper the caster's ability to accurately target spells that require attack rolls, but otherwise their casting ability is unaffected by fatigue.
The game represents tiredness with the fatigued/exhausted conditions, and these are seemingly representative of physical, rather than mental, exhaustion; though they hamper one's co-ordination and strength, they don't seem to have any impact on a character's mental capacities. An exhausted character has no problems concentrating, recalling knowledge, interacting with others, etc. This is obviously not a perfect reflection of what tiredness does to people in the real world, but nonetheless that's the rules as they are.