In General
The typical recommendation here is to decouple experience from the process of killing, and take advantage of the concept of encounters.
In other words, an orc isn't worth X experience points, a standard fight with an orc is an encounter that is worth X experience points.
You build an encounter using combat enemies, traps, skill checks, and so on. If the players successfully resolve that encounter, they get experience appropriate to the encounter as designed.
For example, the army of orcs encounter you mention could be designed as:
- A skill check to find an ambush spot.
- A skill check to avoid detection.
- A skill check to set off the avalanche in an effective manner.
You then assign a challenge rating to each part of the skill challenge, and calculate experience based on that. In this instance, the army of orcs doesn't factor into it at all. The fact that they exist off in the background is irrelevant, as is the fact that the PCs killed "a ton of orcs." The encounter wasn't "fighting orcs," so the challenge rating for the orcs involved doesn't come into play.
Another potential encounter for dealing with the orcs is a negotiation:
- A skill check to sneak past the peons.
- A skill check to convince the orcs you have leverage over them.
- A skill check to come to a compromise.
In this situation, NO orcs die. But if the DCs are comparable to the avalanche above, the challenge rating (and experience gain) of the encounter should also be comparable. Orc mortality isn't part of the calculation.
Subverted Encounters
Where things get tricky is when players subvert an encounter. Taking (for example) a challenging combat encounter, and reducing it to an easier skill challenge of some kind.
For example, if the orcs above were a level-appropriate pack intended to fight the party, rather than a massive invading army. The players have avoided a challenging fight, and replaced it with an easier series of skill checks. What happens then?
In general, the advice is to give players full experience points for cleverness. If the players were aware of the encounter, actively worked to subvert it, and achieved a favorable outcome, they should be rewarded.
Not entering the dragon's cave doesn't grant experience, but distracting him while you sneak the goodies out the back door should.
Even if they really got off easy for a particular encounter, you should still be generous with the experience points. What you absolutely don't want is a group that decides to avoid wacky hijinks because they result in an experience point penalty.
Here There Be Dragons (that are easily circumvented)
The exception here is when players are able to easily circumvent encounters ALL THE TIME. Either through the exploitation of high-powered abilities, poor encounter design, or other factors.
If everyone is having fun, don't worry about it. But if players are starting to get bored, you may need to change things up:
Have a chat with your players.
Be a bit stricter about the consequences of improvised actions (how often do you hear about attacking armies being destroyed by avalanches historically?).
Look up how to counter obnoxious abilities.
Use harsher victory parameters (the dragon isn't just sitting on his loot... He actively intends to destroy a nearby town!)
Wait, You Want to Fight What?
Throughout this answer I've talked a lot about the intent behind the encounter. But what happens if you hadn't intended for the players to encounter the orcs at all?
The ideal is to sketch out the encounter ahead of time. Listen to the players' plan, figure out some rough DCs, hidden obstacles or surprise twists, and CRs. Then calculate the experience point potential before the players even start.
The other way is to make something up roughly commensurate to the challenge (plus a reward for creativity) after the fact.
Throw the Whole Thing Out
Another alternative is, of course, to do away with experience points and simply level the group periodically (every N sessions, at key points in the story, etc.). This works particularly well for groups that like walking around the encounters, because it avoids the entire debate about what is or is not worth 50xp.
