Creating a golem requires binding an earth elemental to it. It is akin to enslaving an intelligent being (the stupidest of them, small earth elementals, have an intelligence of 4 , minimum for sentience is 3). But creating one is not considered an evil act. Why?
Because not all elementals are considered intelligent creatures
The rituals don't describe what kind of elemental spirit is bound when creating a golem, so we could assume that any elemental is valid, even the smallest of them are valid. Checking the main four elemental types, Air, Earth, Water and Fire, they all got only 4 intelligence.
Magical Beasts are similar to animals but can have Intelligence scores higher than 2 (in which case the creature knows at least one language, but can’t necessarily speak).
A Mudman, on the other hand, has no intelligence score, being merely something like chaotic constructs of pure elemental power that has the bare minimum instincts to be able to defend it's own existance, as described under their flavor:
Mud pools are formed where the Elemental Plane of Earth and the Elemental Plane of Water commingle in the multiverse, and thus mudmen are born. Occasionally, a vortex opens to a region on the Material Plane where magical waters have stagnated against the land, thereby forming a mud pool. Mudmen pass through this vortex to the Material Plane. Though not evil, mudmen look with disdain on any who trespass in their mud pools.
Which makes you wonder why they don't have at least 1 intelligence. I guess they just aren't bright enough even to realize they are alive. But it makes you wonder if there aren't other elementals like a mudman that has no intelligence.
However, considering the most common elemental used are earth elementals, we have to take a look at what 4 intelligence means. Looking at the definitions of Ability Scores:
Can speak but is apt to react instinctively and impulsively, sometimes resorts to charades to express thoughts
Yep, sounds intelligence enough. But what are the examples of such creatures?
Otyugh, griffon, displacer beast
No, all those are beasts with slighly higher intelligence than animals, mostly used as mounts or guardians. So it makes sense that a small elemental can be used as a guardian golem without being considered an evil act, if taming a gryffon isnt an evil act either.
Flavor vs mechanics
Do not be so rigid when reading that text simply because it is mentioned on the flavor of golems that they are created using an elemental, but doesn't mention what type, CR, what spell you can use and how knowing that spell affects the cost of the creation ritual (it would be cheaper if you know how to summon them), so that part of the text has no mechanical weight.
What you are thinking about are those elementals that the game calls as True Elementals, which are spirits inhabiting elemental matter, which can be summoned and bound by spellcasters. While the game says that the statistics of such elementals are always the same for their size (see earth elementals), it doesn't really list all possible sizes, missing Tiny, Diminutive and Fine creatures.
Knowing that, elementals are described as coming in a variety of forms:
These creatures appear to be little more than living clumps of elemental material but take many shapes and sizes, and in many cases, when the term 'elemental' is used, it refers to one of these creatures.
But even more specifically, earth elementals mimic the Material Plane life forms:
Like other elementals they come in a variety of forms often imitating Material Plane life forms.
So why are elementals limited to small size if we have cats, birds, and even insects in the Material Plane. The bestiaries are not an exhaustive list of all creatures that exist, as such, there may be tiny, diminutive and even fine earth elementals that we never heard about.
In short? Racism
D&D (on which pathfinder is based) was extremely racist. Or maybe the term is speciesist? I don't know. What I do know, is that if you kill an Elf, you're an evil murderer! But if you kill an Orc, you're a hero! And killing an elemental? eh, who cares?
It's a fact of life in the D&D world that the moral standing of a being depends on its race. You can freely rob, kill, enslave, and do pretty much whatever you want to without being evil, as long as you don't do it to one of the "good" races. This even applies if you are a member of that race; a Drow killing other Drow is just something that happens, no more significant than eating a sandwich, while a Drow killing an Elf is a severely evil act.
Depending on what you believe, the morality of an action is either determined by society, the gods/god, or is some fact of the universe. Whichever it is though, applies to pathfinder. Society doesn't care, the gods don't object, and the rulebooks (the facts of the universe) say it's fine. Q.E.D.
Keep in mind, that the fictional world you are playing in is not the real world, and the rules of the world are different - whether those are rules of physics, or of law, or of morality. Just because something is evil in OUR world, doesn't mean it is also evil over there.
Just consider, there are people in our world who believe that eating ham is a sin. Is it a sin in your game world? Ask your DM. Maybe ham is fine, but anyone eating chicken is immediately pulled into hell.
Trying to evaluate your actions in-game according to real world morality simply doesn't work. The entire game is built around killing things and taking their stuff, and if you're going to insist on trying to act according to our modern sensibilities, all it's going to get you is shanked by the next goblin you meet.
So, in conclusion? Enslaving elementals is not an evil act, because it is a different world, with different moral rules, and in that universe the moral rules say it is not evil.