The typical approach that I have used and played under is this:
Tier-4 classes can gestalt with tier-5 classes
Tier-3 classes can gestalt with tier-6 classes (i.e. non-adept NPC classes)
Tier-2 and tier-1 classes cannot gestalt
It’s not perfect, but it seems to work pretty well. The higher-tier classes are still more powerful, even with these benefits, but the lower-tier classes are much more interesting than they usually are, giving players more incentive to play them and more reward, in terms of fun, for doing so. My current character is a smorgasbord of monk, paladin, and spellthief, and I’m having a lot more fun than I would as a wizard, even though the guy who is playing a wizard can ultimately do a lot more than I can.
As for distinguishing between classes, the tier list itself covers most classes, and why each class is in its tier explains how to eyeball other classes.
Tier 1 and 2 are relatively easy to adjudicate: if a class has access to something at most levels that trivializes problems in an overpowered or broken way, it’s tier 1 or 2. If it doesn’t, then it’s not. Generally, this means full-casting: a spell progression that eventually hits 9ths. The cleric, druid, and sor/wiz lists have many, many entries that fit this description. The shugenja and wu jen lists are not quite at that level, but probably have enough. And artificer flies here just because his crafting means he can use any and all of the above, if he’s careful.
It’s important to note that tiers should be level- and optimization-agnostic. A broken 20th-level capstone won’t change the tier of an otherwise-fine (or otherwise-awful) class.1 A particularly broken build that manages to break the game with some shenanigans doesn’t count either.2 The idea is that something about the class itself is, at most levels, overpowered.
The difference between tier 1 and tier 2 is, basically, the difference between prepared and spontaneous spellcasting: can the class change what it’s doing regularly, so that it can cover all sorts of very-different situations, or does it get locked into one set of abilities that, while extremely powerful, leave some room for flaws and vulnerabilities?
The other tiers are somewhat fuzzier. Tier 3 is defined as being excellent at one thing and reasonably competent at other things (e.g. warblade), or being quite good at a lot of things (e.g. bard). Tier 4 is decent, or even great at one thing, but useless outside it (e.g. barbarian), or OK at a lot of things but struggles to excel (e.g. rogue). Tier 5 is OK at their specialty, but not even the best at that and utterly useless outside it (e.g. fighter), or is just so spread out that it struggles to get much done (e.g. monk). Tier 6 is NPC classes (and the Complete Warrior samurai, which is basically a fighter with his feats pre-chosen, and chosen poorly). The truenamer doesn’t get a tier, since it doesn’t even work without a lot of work (a high-optimization truenamer probably functions around tier 4 or 5 in a mid-optimization group).
cf. truenamer, who goes from nigh-unplayable to quite-possibly the most inherently-powerful class in the game when he dings 20th.
cf. Pun-pun’s use of ex-paladin