Ban the Player’s Handbook
The Player’s Handbook was the first book Wizards wrote, and it has numerous severe flaws, which Wizards themselves acknowledged over the years. The Dungeon Master’s Guide and Monster Manual have some similar problems, though none as serious. Other early books (most notably Complete Warrior, the first supplement) have similar problems, but are probably workable aside from the base classes.
In so doing, you eliminate a number of the classes with the biggest problems in 3.5: cleric, druid, sorcerer, and wizard, on the overpowered side, and fighter, monk, paladin, and ranger, on the underpowered side. The archivist (Heroes of Horror) and artificer (Eberron Campaign Setting) should probably be banned as well on the overpowered side, though honestly without Player’s Handbook spells it may not be necessary. For underpowered classes, there are many (many early base classes were modeled on the fighter, monk, paladin, or rogue), but more on that in a moment.
The general goal here is to ban traditional core classes, because the mundanes can only deal damage, but can deal too much of it. Meanwhile, the spellcasting classes are so good that only dealing damage is a bad strategy, and any nerfs to prevent one-shot-kills are only going to make that worse. So we throw out both, and turn to supplemental material for more balanced options.
Replacing Magic
The problem with spellcasters, more than anything else, are the spells. You probably could use the cleric, druid, and wizard in a relatively-balanced game just by banning Player’s Handbook spells, but ultimately their ability to completely change their load-out from day to day is extremely powerful, so we should try to avoid it.
Beguiler (Player’s Handbook II), dread necromancer (Heroes of Horror), and warmage (Spell Compendium) are much better-designed arcanists than the sorcerer or wizard. Each has a particular theme, and they are good at that theme but unable to do everything. That helps both add variety to your world, and limit characters of these classes.
On the divine side, the Complete Divine base classes, favored soul, shugenja, and spirit shaman, are workable. Without Player’s Handbook spells, the shugenja might have some trouble; giving it spells from Spell Compendium will help, however. These classes are somewhat dubious, however: they’re closer in mechanics to the overpowered classes from Player’s Handbook than I’d like. There aren’t analogues of the beguiler, dread necromancer, or warmage for divine spellcasting, however. Miniatures Handbook has the healer as a low-power divine spellcaster, but it’s not really all that good at healing and it’s good at absolutely nothing else, so that’s not an ideal choice.
In addition to arcane and divine spellcasting, supplements add many new forms of magic, most of which are more balanced than in Player’s Handbook. Expanded Psionics Handbook, Magic of Incarnum, and Tome of Magic each add several base classes to go with new magical systems, and in each case all of them save one are quite good. I’d ban soulknife (XPH), soulborn (MoI), and truenamer (ToM) though. The psion might be a little much for this game, but maybe not. The ardent (Complete Psionics) is another option that might be a bit too much but maybe not. The divine mind and lurk from that book are bad, however.
Replacing Martials: Tome of Battle
Initiators are much more flexible, mobile, and versatile than traditional damage-dealers, which is good, but do not achieve the same damage levels, which is perfect. An initiator can be powerful – often considered better than traditional melee – despite not one-shot-killing everything. This allows you to have more interesting, varied battles that don’t end as soon as someone manages to get into position long enough to full-attack, and these classes can stand alongside the above magic classes better than traditional mundanes classes.
Prerequisites
Without Player’s Handbook feats and spells, it may be impossible to meet prerequisites for feats or prestige classes. These should simply be handled on a case-by-case basis. For the most part, you can probably safely eliminate a lot of feat requirements; many are simply taxes that weren’t necessary in the first place. Otherwise, switching out requirements or allowing certain Player’s Handbook feats to be taken will work.
Results
Without base classes that have bonus damage dice, without Power Attack, without Rage, and so on, martial characters will have to rely on the Tome of Battle maneuvers for damage, which means their damage will be kept inside fairly tight guidelines. On the other hand, they will be more flexible and more capable of dealing with a variety of situations. Meanwhile, spellcasters will have greatly reduced access to the most powerful spells, and their mechanics will not be as flexible, preventing them from overshadowing the warriors.
The biggest problem is the lack of mundane ranged attacks. Tome of Battle is almost strictly melee-only (there are enough maneuvers that don’t require a melee attack that you can make a Tome of Battle archer, but you have to be very careful in your selections and ultimately while you could make one, you probably couldn’t make a second one that was much different from the first). “Gish” archers, combining ranged weapon attacks with some form of magic, will work well, but that is not quite the same thing. The best solution for this, I think, is to use one of the many homebrewed ranged-weapon disciplines out there.
The other obvious problem, perhaps more serious, is that things are not all in one neat book, but rather spread out across many books. Feats, in particular, are problematic because Wizards never published a Feat Compendium. This will be difficult for groups that lack fairly-thorough system mastery, and may be impossible for groups without a large library. That said, if it’s an option for you, I think the game works much better this way.
My Experiences
Most of my games fall into these lines by simple gentleman’s agreement; this is how I usually play the game. My groups have rejected most traditional damage-dealing mundane classes as boring and limited, while also eschewing the worst abuses of caster dominance and avoiding one-shot situations.
I have also played in games that have made these rules explicit, or even gone further. One of the best games I’ve ever played in was strictly Expanded Psionics Handbook and Tome of Battle, nothing else, in a vaguely Oriental setting where psionics were meditations and maneuvers were martial arts, frequently blended together by roving would-be masters.
So these rules have allowed our games to function despite the fact that we, as players, push the system very hard. We optimize fairly heavily (very heavily by the standards of this site) as a matter of course, but with these rules it’s rare to see one-shot kills, even at relatively high levels.
Pathfinder
Pathfinder lacks an analogue to Tome of Battle, and as such I think it would be very difficult to achieve the same effect. Dreamscarred Press is working on their own version, Path of War, but it is as yet incomplete, so I do not know if it will achieve the same things. Moreover, much of the Pathfinder supplemental magic classes have focused on the same mechanics as the core spellcasters, which means many of them have similar problems. I’m not sure if banning core spells will help Pathfinder; I know Spell Compendium had far more balanced spells than Player’s Handbook but I don’t know that the same is true for Pathfinder’s analogues.
