I can't find the exact rule on the Pathfinder SRD, but there's a link to a topic here that describes it:
And this is the rule in the D20SRD, which I'm pretty confident is the same:
You might find the exact rule in your rulebook somewhere (but I don't have one to check, unfortunately)
It essentially comes down to this:
Every other square of diagonal movement is counted double. So the first square is 5ft, the second is 10ft, the third is 5ft, etc. This approaches the actual diagonal reasonably over short distances.
So in your example, the distance would be 10ft going straight (30ft-20ft) and then 20ft going in a pure diagonal (the lowest of the two) with every other square counting double; ie the distance multiplied by 1.5x, which gives 30ft of diagonal movement for a total of 40ft travelled.
The rule you remember of "take the highest" is from 4th Edition D&D.
(As you can see the final result comes within 5ft of the trig answer, so it's accurate enough for game purposes)