This is just a logic question, but why does having a familiar/mount reduce your leadership rating? IMO I would be more inspired/ would have more morale to follow someone on a mount/familiar. It just makes no sense to me.
There's no RAW answer to this, and probably no RAI that can be supported, because the authors simply haven't said. There are, however, some clues that can point us at something both likely and sense-making.
The most important of these, in my opinion, is which list that penalty is on. Leadership has 3 separate lists of penalties to your leadership score. One applies to everything, one only when determining followers, and one only when determining the cohort. Only getting a cohort (representing a close, loyal friend with a significant share of your attention) is affected by having a class-based companion creature (also a close, loyal friend who, forgotten familiar syndrome aside, is at least nominally expected to receive a lot of attention from the character). To me, this strongly suggests spreading your attention too thin and cohort jealousy as likely mechanisms.