If a merchant hires the PCs as guards, how much should the PCs earn?
Rules guideline #1: Treating them as "trained hirelings"
According to the Goods and Services rules, a trained hireling (including a mercenary guard) is expected to make three silver pieces per day in wages. The PCs are, at the very least, worth this much, even if they're level 1 adventurers with no reputation to speak of.
However, the rules note that:
This value represents a minimum wage; many such hirelings require significantly higher pay.
Thus, you shouldn't use this wage as any kind of final word on how much your PCs should make, just as a kind of sanity-check guideline for how much a normal merchant caravan would expect to pay normal mercenary guards under normal circumstances. D&D, of course, is designed to be a game about exceptional people (the PCs) in exceptional circumstances (your campaign), so they're likely to be an exception.
Rules guideline #2: However much gets them to WBL
D&D 3.5 is heavily balanced around the PCs having approximately the correct amount of stuff (magical items, consumables, access to spell effects, etc.). There's a table on page 135 of the Dungeon Master's Guide that provides guidelines for how much total wealth a PC should have at a given character level. So, if you're not sure how much to pay them, you can't go too far wrong just paying however much will bring them up to the appropriate Wealth By Level entry.
Not in the rules: It doesn't matter, adventures don't run on wages
(This isn't a rules-based answer to your question, but a bigger-picture piece of advice about running a D&D campaign.)
How many fantasy novels have you read where the protagonist(s) were hired to guard a merchant caravan, trundled along for a while, fought off a bandit attack or two, got to the destination, were paid the agreed-upon wage, and that was that?
I'm guessing the answer is zero. Fantasy novels (and D&D campaigns) aren't about that kind of story. That's not even a story, that's just a day at work! In D&D, here's what happens when you get hired to guard a merchant caravan:
- The "merchants" turn out to be doppelgangers who have replaced the real merchants and are smuggling contraband into a neighboring kingdom as part of a sinister plot to overthrow the monarchy. You overhear them plotting one night, foil their evil plan, and are rewarded by the Good King Whatsisface with your choice of weapons from the ancestral armory.
- You are ambushed along the way by a rampaging dragon, which eats the merchants and destroys the caravan. You are forced to flee and warn the guards at the first town you come to, who request your assistance fending off the impending dragon attack. The "caravan guard" thing was just a segue into the real plotline of the adventure.
- Your murderhobo PCs can't be arsed with guarding a merchant caravan when they could just kill them and take their stuff. You try and dissuade them but they are unmoved. Their "wages" are whatever the merchants were carrying.
In other words, it doesn't matter what the merchants agree to pay them. Just pick a number. Unless you're running a pretty boring campaign, something more exciting (something that comes with other opportunities for rewards!) is going to happen before the caravan gets where it's going anyway. If you, as the DM, weren't already planning on pulling something like this (which you should be), your players will probably do it for you.