My old DM used these methods when he wanted to show that a power was overwhelming:
- Have the enemy appear in overwhelming numbers, e.g. hundreds of anhkegs, a dragon with eleven more dragons circling in the sky above, an entire army approaching from the distance. Or, use a single enemy who is clearly unstoppable, like a great wyrm or epic level creature.
- Ask for a skill check to reveal the enemy's strength—Sense Motive, Knowledge (arcana) or whatever is relevant. Success reveals that the enemy is way out of your league.
- If the PCs still refuse to flee, have the enemy defeat them, but captured alive. They may have to escape, or be forced into some quest. This will teach them that they can't win every fight without a TPK they weren't expecting. The second time they make the same mistake, they've had their warning, so don't be so merciful.
Some other advice:
- From The Strategemata: "Scipio Africanus used to say that a road not only ought to be afforded the enemy for flight, but that it ought even to be paved." Give your PCs a clear opportunity to flee unharmed and continue the adventure by another path.
- Make fleeing the encounter a challenge in its own right. The encounter is now about flight, not combat. The PCs may have only to reach some safe house (think Left 4 Dead):
- Undead can't enter a consecrated place.
- Vampires can't cross running water.
- Huge creatures like dragons can't fit through a small dungeon entrance.
- An army may be unable to cross a national border or enter a city.
- The PCs only need to hold out or hide for a fixed time, e.g. until reinforcements arrive, or until sunlight forces the undead to retreat.