I have never GMed pathfinder, but I understand it is extremely similar to DnD where I have a fair bit of experience.
Make it fit the encounter
I generally try to make the treasure fit the encounter in the sense that the treasure should be what the creature would logically have. Under most circumstances that means I totally ignore both what the players want and the wealth by level as well as the guidelines.
So, the phase spiders you mentioned would likely have nothing but their venom. A group of goblins would have some simple spears and some simple bows and arrows. For things like that, loot is fairly simple to come up with on the fly as "What would they have?"
A group of full on humanoids is more intersting as those tend to come with equipment, but then the group finds the equipment they were using pluse whatever loot the bandits collected so far. That of course is a bit harder to come up with on the fly, but I'm still being guided by quickly thinking of what equipment did I have them use and what would they likely have.
Now of course, someone might come back by pointing out (as some players have with me) that those threats can kill adventurers and might have some adventurer equipment, so occassionally I sprinkle in something else, but then I literally just make it up on the spot and it is normally generic and low level (after all, if the adventurer that encountered these things before was powerful enough to defeat it or even get away, that equipment wouldn't be there now, so the equipment they find from random encounters will always be much weaker than what they have.)
I think that scheme has the advantage of keeping things realistic and for most encounters, especially the occassional random encounter, and makes loot easy to generate on the fly. But it has the horrible disadvantage of making meaningful loot from that kind of encounter nearly nonexistent and would leave them way behind in the wealth by level charts.
But when I GM I generally expect the players to get most of their wealth through quests. Most quests will come in the form of someone hiring the band of heroes and paying them very well for their services. If the employer/quest-giver is well connected the heroes might be able to directly bargain with the employer to get them a hard to acquire item they want in place of some of the gold initially offered.
When the story says that the quest-giver can't pay them well because they are poor themselves, then I have the quest giver reward them with information which leads to another quest which has a direct payoff. For instance, an old widow's son has been turned into a lycanthrope. The heroes rescue him and turn him back. The old widow can't pay them directly, but she knows the old local story about the Archmage Generet from over 200 years ago who died leaving behind his tower. The tower is reputed to hold his great wealth but remains home to the remains of his experiments so it is too dangerous for any of the locals to go to.
The loot from the tower will fit with what an archamge would likely have lying around and is meant to compensate for the work on the lycanthrope so it will be far richer than the threat level of the gaurdians would suggest by the book. But I still fill it by thinking mostly about the kinds of things an archmage would have lying around. They will find things like potions, wands, components, and scrolls in abundance, but probably not a set of plate mail. If they want a set of enchanged plate mail, they will either sell off the loot to buy it or else they will start asking around specifically for a lead to that sort of quest.
Getting them what they want
Which brings me to leads. In games I run, the players buy most of the equipment they specifically want. But if they want something truly exotic where buying it won't work and they can't negotiate for it from a quest giver, then they need to look for a lead. A lead is normally the start of quest meant to bring them to exactly what they want.
So, the Paladin needs a new set of armor, but not just any set, he wants a legendary one. His player tells me ooc, and in character starts asking around. No one can sell him something like that, but eventually someone will tell him the story of Sir Gawain the Kind who was killed by the dragon Karlesto. Sir Gawain had a set and it likely is now in Karlesto's possession. No one is paying to have the dragon killed because it has been inactive for decades...but if they do kill it they will likely find his set of armor along with the gold the dragon has been hoarding.