I think the best way to anger the PCs without angering them is to do stuff to things they care about, rather than them. That way they aren't going to fell that the universe, and by extension you, is picking on them. Anger them by having antagonists trample over something they really care about.
Do your PCs have family? Have an antagonist kidnap or kill them. Or, more subtly, have an antagonist destroy their lives (by throwing them out of their home, for example). When the player finds out, this is likely to upset them.
Your players may not have a well developed backstory, in which case you'll have to work a bit harder. Is your player a paladin? Have the antagonist defile their God's temple. A hoarding thief? Steal their stash. Someone with a really nifty or signature weapon? Steal or break it (as long as there's some prospect of repair, the latter probably won't rile them, or the antagonist could have the only equivalent widget, which may have been their motivation in destroying it.
My point is (a) think about what they value (b) be creative (c) make sure that if you're taking away from them that you're ready to give back too - it may not be necessary immediately, but there needs to be a sense of progress as a result of the storyline you're planning.
Speaking personally, I love it when the GM is mean to me, so long as it creates story. If your players are sufficiently mature they will hopefully realise that what you're doing is good, but it's your responsibility to make good game out of it.