The Simplest Solution
Tell them when the encounter is over. Then don't let them try to revisit that encounter.
Geting there in the first place
In order to have it be an issue in the first place, the players have to have been rewarded for it at some point.
Several techniques lead to this perseverant behavior:
- allowing multiple rolls to find things
- having every encounter be worth XP
- Only giving Encounter XP if the players find the macguffin
- fudging die rolls.
Getting away from it
Adopting the following table conventions really helps:
- Let It Ride
- Dice are Players, too
- Bonus XP for not using Player Knowledge
- Low-XP encounters
- Color Encounters
- Play the Psychology of the Character
- Non-XP rewards
In detail...
Let It Ride
Nothing gets rerolled unless the rules specifically call for it. The first game with such a rule I encountered was MegaTraveller, tho it wasn't so explicit; Burning Empires was my first contact with it as an explicit rule.
A Roll to find something, for good or bad, convinces your character of the existence or non-existence of that thing. Fumble a search roll for a non-extant secret door, and you're convinced there is one that you don't know how to open.
Dice are Players, Too
This is an old concept, but a new wording for me. The dice are there to add something to the game. Accept their input, as it leads to a better game most of the time. If you go to the dice, always be willing to accept their input; corollary - if you have reservations about the outcome, don't go to the dice.
Bonus XP for not using player knowledge
The moment you award someone publicly for not using Player knowledge that the character shouldn't have, you start to break that reward cycle. Unfortunately, this one needs to be VERY consistent, due to the "random reward effect"... It need not be much, but it should be just enough to be noticed. In BXCMI D&D, I considered this "half a bonus" (an XP bonus is 1/20th of a level per the Cyclopedia).
Low XP Encounters & Color Encounters
If you put in encounters with John the Crier, and he's not a threat, has almost no useful information, but adds color and verisimilitude, players get used to these small guys. If you reward interacting with them, players will tend to do so.
For 1st level D&D, a 10 XP encounter isn't much, but they add up, and having several quick, colorful, low XP encounters can make a game much more story and much less tactical wargame.
Likewise, not all scenes need to have something to do. It's fine to occasionally describe a room just to set the feel of the place. For example, if the entrance to the dungeon is a crystal staircase down a hole, it's a huge difference from a steel one, or a hewn rock one. Describe the nature of the mold on the walls, and the moss by the door. Don't waste time, but do add just enough to set a tone in these color encounters.
Likewise, some encounters are with intangibles. Arriving in town, and everything reeks of an acrid scented effluent, instead of the normal sulfurous effluent, describe the difference. Nothing to do about it, but it provides information... "something's amiss here, because the sewage smells wrong." Giving it as one line like that is a clear warning. "Ah, the smell of the city. Baking, tanning, effluent in the open sewers... There's an acrid tang, and the sewage isn't quite as rotten-egg a smell as one woud expect." (I'd be worried, because the implication is either dysentery or vomiting... if you can smell it in the sewage, in large numbers.) Allow the knowledge rolls to understand it, and move on. If they don't make, just move on.
Play the Psychology of the Character
When a player decides a character has some psychological element, reward playing it. Don't punish non-play, but reward the play of it. Sometimes, this means reducing color text to simple narrative, as the group doesn't care, but the player knows the character should.
For example, if playing a merchant in a tunnel-questing game, reward the player for "I'll check the local assay office for prices", while in a merchant game, the player might need to actually spend time interacting to earn the same bonus.
Likewise, if the character "has no interest in frogs," and ignores the Frog God statue of Gold, reward the player. Sure, it's valuable, but the character doesn't care, so leaving it be, especially when this results in not finding some clue, is worth reward.
Non-XP rewards
Not every reward needs to be experience points. Character cash can be a reward. Magic items. In games that have them, fate points or hero points.
Heck, one of the best rewards is a small token food reward. I once experimented with rewarding players for clever actions not with Die-Roll mods, but candy. Got some really clever actions.
Even a "Retry Token" allowing retrying in the face of Let It Ride is a good reward. (Especially in systems with explicit LIR rules.)