The answer dates back to the early days of the hobby. Sherman, set the wayback machine for 1975...
In the original D&D game, with only 3 classes (Fighter, Mage, and Cleric), the idea was that only heroes and villains had classes. (It wasn't until later that thieves as "heroes" was added.) NPC's who were not one of the PC classes, including the vast majority of fighting men, were either level 1 fighters or "level 0" characters, with HP but no ability to fight (Man-1).
In the fanbase, People added all kinds of specialized classes. (We're still talking before 1979.) Each with some measure of combat progression and specialized powers and/or spells.
Only two made it into AD&D, namely the Bard and Ranger, tho' Gygax and Arneson added a couple more themselves. (Gygax added thief, druid, and paladin, Arneson added Monks and Assassins.)
AD&D defined that, axiomatically normal people lacked classes, period. They fought as if level 0 fighters, they had HP by job or GM fiat, in the range 1-8+ConBonus... and saved as level 0 fighters.
Again, people added "NPC classes" - less effective than the PH classes, per XP, but still, they proliferated. None were official, but they were in Dragon, and that was good enough for many.
2E said the same things, but added a book of specialist classes later on.
3E comes along, and again uses the same core classes... but defines that they are not the majority of people.
Over time, the level of rulers also changed... Gygax has oft been quoth as having said kings were 1st to third level fighters in Greyhawk... Later published versions showed 6-9th level kings. Later still, 12-15th level.
In 3E, the king might not be a fighter at all... but he's almost certainly not a warrior.
In the 3.x DMG, you'll find a process for populating towns. This was a strong step away from the roots, and into the branches, of the D&D tree... those same ideas that, with enough classes, everyone could have class and level.
But, in order to have PC classes stand out, most people had to be weaker... hence a return to the old saw of NPC classes. And in the world building guidelines, about 90% of the population are in those classes. And those classes are only slightly weaker. (Actually, they're quite playable as PC's... and adepts are not too underpowered.
Reasons to use them are simple: they leave the number of really dangerous mid-levels less than capable of pushing around mid-level PC's. This makes generating them both simpler, and detailing out a population slightly easier. They're for your hired hands, farmers, wrights, temple staff, and the town watch. (Note: the watch leaders are fighters, and the priests are clerics... but the average joe is a warrior or adept.)