Mostly, yes.
Let's try to make sense of why, shall we?
The description states that your character is "not merely intolerant — he is outspoken about his beliefs and may actively work against the target of his prejudice" (pg. 82).
In other words, the thing that makes this an out-and-out disadvantage rather than simply a "roleplayed" personality trait is that you don't just have beliefs, you also can't help but act on them – irrationally.
So, why the heck would it affect negotiations that way, mechanically? Well — what happens when you're obviously hostile to someone during a negotiation?
They think to themselves, "Why are we negotiating at all?" — clearly you really need this thing, if you're being forced to overlook your obvious and palpable distaste for their very existence.
You make slip-ups against the normative rules of decorum, which can require some costly concessions to patch up: your underworld contact demands all the money up front, the troll clerk smuggling your package through customs demands a second bribe not to "lose" it, &c.
A lot of people enjoy screwing over people who hate them, even if it might cost them a bit of business or whatever. "Oh, sorry, that service costs double for Humanis members. Special new illiteracy tax. You can read all about it in this elvish newspaper. [wink]"
Partly, I would guess, it's a mechanical tax to encourage you not to negotiate with the target of your prejudice unless you have to — bully, threaten, steal, or kill instead.
If that doesn't sound like what you picture as "Prejudiced," my recommendation is… don't take the mechanical quality. It's clearly not a good fit for the character in question.