This is a very basic question, but I'm a very new DM and I can't seem to find any indication how characters acquire mundane weapons and items beyond their initial equipment. I know some items they can discover in treasure caches, and I figure I could put weapons there as well. But can players buy weapons? Do they need to find a merchant selling the weapon they want or do they just deduct gold from their holdings and assume the items?
This is entirely up to you as the DM!
Typically, after each session or adventure, the characters return to some sort of a town. Depending on the size and wealth of the town, you can limit the availability of certain items. If you run a more casual campaign, you can just let players deduct the gold from their coffers and purchase the item without any roleplay.
Keep in mind, however, that limited items can be an important plot point. For example, a journey through the wilderness can be made much more realistic if you point out that they won't have any stores to buy or sell from on their journey.
Sorry to drag you back to the real world, but think of how you acquire items in the real world ... players can buy whatever they like (within reason) assuming they have access to a source.
So if you need a piece of rope ... you go to your local hardware store, or you pop next door to a friendly neighbour and see if they have some you can buy (lend).
But if you're at the bottom of a cliff, trying to get to the ruins of Kazak perched high up on a ledge, then you hope someone planned ahead ... or you keep your fingers crossed that the merchant caravan you passed earlier in the day has got some.
Neither answer hits the RAW approach here so let me give it to you:
The DM guide and the RC are largely silent on how purchasing works beyond some general guidelines and this is intentional, they leave that up to you. However, there are several built in RAW ways to acquire magic items in 4e and a lot of it depends on setting, enchantment, and item rarity, let me address all of these things.
Setting: you didn't mention your setting so I assume its PoL so the full range of items is likely available somewhere, although you might limit things from the Feywild or Dark Sun. This is up to you and what you want to include in your world. IF you were playing DS say then this would obviously be more limited by your setting.
Enchantment: most mundane things should be able to be bought in shops in any reasonably sized town your PCs encounter, typically mundane equipment cannot be sold back, if sold probably 20% is the rate it should be sold at. What is available at any given time in any given shop is completely up to you and your digestion. You can choose to abstract shopping to just gold deduction, or you can make it more complex by having your PCs go to shops and even haggle, depends on how much time you want to put into it, and how big of a deal you want to make it. Your PCs will likely only have the wealth for items in their level range, but you could also make sure they don't get overpowered weapons by limiting shop inventory (sort of like video game RPGs sometimes do)
Rarity: This one is again up to you, but most magic items in 4e now have a rarity attached to them, based on that we know something of their availability
Common: Available for purchase, maybe not everywhere, but if your PCs look hard enough they should be able to buy a common item. Sell back or reduce to Residuum at 20%
Uncommon: Generally hard to find, you might let your PCs purchase uncommon items, but you should probably make them work for it (our DM makes us special order them from a specific group, or jump through similar hoops). Sell back or reduce at 50%
Rare: Incredibly hard to find, usually quest specific, but can occasionally be found in treasure drops (I'll get to treasure drops in a minute). If you want to allow players to buy these I'd do so cautiously and make it incredibly difficult. Sell back and reduce at 100%
Now, you mentioned the possibility of including magic items/weapons in your treasure drops, if you aren't already doing this you might want to look at how your doing treasure. If you're not rolling treasure parcels I highly recommend starting. There is a table in the back of the Rules Compendium that gives you 4 lines to roll treasure, this allows you to create random treasure parcels tailored to your PCs' level. The last line of this determines the magic items dropped in the parcels, it even builds in item rarity (common is an odd roll, uncommon even, rare 20). Some suggestions:
Wizards highly recommends player wishlists so that the DM can appropriately fill out the treasure parcels with things the PCs want/need. This is a good plan if your players know what they want. Also if you're rolling the treasure parcel ahead of time then this is the way to do it.
The way I've been doing it in my game is actually to roll the treasure parcel at the end of the encounter and if a magic item is awarded the group decides who receives it (we do it randomly with a bit of fairness added, but it can be a group decision based on who needs it or whatnot) and then they go home and select their item from all available equipment for the next session.
If you want to include mundane equipment or weapons into a treasure parcel add it wholesale or substitute it for some of the gold.
Finally, keep in mind that the math of 4e is heavily dependent on your characters having the proper equipment, if they don't they will be ill prepared to face the challenges you present. A PC should pick up a new magic weapon or implement ever 6 levels or so (this is the rate at which the enhancement bonus changes), and a new set of armor and new neck slot item at about the same rate. This can come from either finding that item, purchasing it, or using gold to enhance their current enchantment through the
Create Magic Item Ritual
Hope this helps expose some of the RAW mechanics of treasure awarding in 4e and that you have a great time.
A TL:DR for those just looking for the answer to mundane equipment:
Mundane equipment should be available in most towns, whatever kind of weapons/equipment they'd like to have, you can make some exceptions, but generally PCs should be able to find/acquire anything they need that isn't magical. If you want to put an emphasis on it then make them find a shop and haggle a bit, if you don't care then just say, "yes you find a shop, deduct the purchase price." Something harder to find you might want to make them go on a quest, or find a wizard or something. It's mostly up to you.
If your not giving intrinsic bonuses and your PCs are looking for mundane weapons after the first level or two something is wrong. (See my 'graph on math above).
Optionally, hand wave the whole entire thing and let your PCs use just in time purchasing. They can have the item they need when they need it, but need to be capable of purchasing it. (This is a great way IMO to assume competent adventurers)