This question is about creatures that don't have a special climb speed or swim speed, what is the case for most playable races.
The general rule is to make a climb/swim check and move at quarter/half your speed if the check is successful. But some bonuses to your base speed seem to be ignored.
Example :
Expeditious Retreat
This spell increases your base land speed by 30 feet. This adjustment is treated as an enhancement bonus. There is no effect on other modes of movement, such as burrow, climb, fly, or swim. As with any effect that increases your speed, this spell affects your jumping distance (see the Acrobatics skill).
There are a lot of other ways to get better land speed : alternate racial features, class features, feats and magic items. Some harmful effects from many types of attacks, including physical and magic, and the major drawback One Leg give penalties to it.
So, as a general rule, what types of bonuses and penalties to your base land speed are taken into account in the calculation of your climbing and swimming speed ?