The Pathfinder rules for preparing divine spells has this to say about the time at which prayer occurs:
[A divine spellcaster] chooses a particular time of day to pray and receive spells. The time is usually associated with some daily event. If some event prevents a character from praying at the proper time, she must do so as soon as possible. If the character does not stop to pray for spells at the first opportunity, she must wait until the next day to prepare spells.
In short, you can’t change your time, but you don’t lose out on your opportunity to prepare spells if you cannot make the appointed time.
However, be aware that in many settings, particularly those coming from Dungeons and Dragons, the time to pray for spells is set by the god/religion, not the spellcaster. For example, sun gods will typically have you pray at high noon; gods of darkness and secrets tend to like midnight. It’s usually supposed to be whatever time your god is most powerful, though I could see, say, a sun god wanting you to pray at dawn (for the return of the sun, triumph over the dark, etc.) or at dusk (to see you through the night, maybe).
So in these cases, it’s really not up to the caster, either initially or later. I could see a god giving a favored champion special dispensation, however, assuming that in your setting it is the god’s choice (rather just the time at which that god is capable of responding with spells, which in some settings it may be). So you should talk to your DM about that / you as a DM should consider that.
Mechanical-balance-wise, the time is completely unimportant so long as you have a minimum of 24 hours between praying because if you can refresh your spells more often, you effectively have more spells “per day.” Spellcasters are generally overpowered to begin with; we don’t need to be adding more on top of that.