If I wanted my party's spellcaster to enhance/enchant a weapon or piece of armor, is there a limit to the number enhancements it can have on it? I haven't read anywhere that there is a limit, but putting an unlimited number on them seems OP. I also remember the game Neverwinter Nights 2 limited you to 4, but it didn't follow the rules 100%.
Magic Item Compendium updated and clarified weapons’ limits. These rules are in addition to those in the Player’s Handbook: for example, Magic Item Compendium does not repeat, but does not repeal, the rule that non-epic weapons and armor are limited to a +5 enhancement bonus.
But for the purposes of non-enhancement magic properties, we have the following rules:
A weapon with a special property must have at least a +1 enhancement bonus. You can’t simply have a flaming longsword—it would have to be at least a +1 flaming longsword. A single weapon cannot have a modified bonus (enhancement bonus plus property bonus equivalents) higher than +10, nor can it have a market price (not counting special materials or the price of the masterwork weapon itself) of greater than 200,000 gp (or 200,000 gp for each end of a double weapon). For weapons that exceed these limits, see Epic Level Handbook.
To add a special property to a shield or suit of armor, the shield or armor must already have at least a +1 enhancement bonus. A single suit of armor or shield cannot have a modified bonus (enhancement bonus plus special ability bonus equivalents) higher than +10, nor can it have a market price (not counting special materials or the price of the masterwork armor or shield itself) of greater than 200,000 gp. For armor or shields that exceed these limits, see Epic Level Handbook.
General Limitation: You have to pay for it
Even self-crafted magical enhancements are expensive. You are unlikely to actually want to have more than +10-equivalent or 200,000 gp worth of properties much before epic levels anyway, simply because it is unwise to sink so much money into a single item. At the very highest levels (say 18th level and upward) you might have considered buying such an expensive item but it is a dubious choice even if it were allowed.
The limit is based on the total enhancement bonus. Each enchantment has, listed as its price, a number, for example Bane has a +1 for its price. A non epic weapon/armor can have only up to a total of +10, and at least +1 of it must be an actual enhancement bonus, and no more than +5 can be actual enhancement bonus.
So for example, a +1 Bane long sword would have the cost of a +2 weapon and has room on it for a total of +8 more on it.
An important note, some armor enchantments have an actual price and not a bonus, these do not count towards the limit.