Clarify the situation
Whenever my players propose a plan that will in no way work, I first assume that I didn't describe things well enough.
Rogue: I sneak into the throne room, steal the document we need, and leave.
DM: (This guy must have a death wish. What am I missing?) The hallway leading up to the throne room is brightly lit by torches at regular intervals. There are columns in the hallway, but the closest one is 15 feet from the throne room door, which is currently open. Two royal guards flank the door. One of them is watching your group attentively. How exactly do you sneak up?
Rogue: Oh, I see...well, I'm a halfling, so I hide behind the fighter. I'll then flip up the hood on my cloak of invisibility and creep up to the door.
In this situation, the player had a different image of the situation in their head. Maybe I forgot to mention the guards or the torches in my initial description. Either way, additional information allowed the player to adjust their plan.
What happens if the problem with a plan is something that the player doesn't know? What if every member of the royal guard has truesight, allowing them to see invisible creatures?
DM: (Crap. He's going to get caught and it's going to feel cheap.) Alright. Your cloak shimmers and you turn invisible. Make me a History check. You grew up in this town, so roll with advantage.
Rogue: (rolls) I got a 13.
DM: (That's not quite enough, but I'll give them something.) A few years back, someone tried to assassinate the king. As a rogue, you heard that the assassin used magic to change their appearance. You also heard that the assassin didn't even get one foot into the throne room before they were cut down. These guards don't bother taking prisoners. What would you like to do?
When giving out information, there are a few options.
- Just tell them. If it's something many people would know, don't hide it behind a roll.
- Make them roll. If you're not sure, have one or more of the party members roll, even the ones who aren't coming up with crazy plans. Maybe the party wizard can advise the rogue about the dangers of truesight.
- Change the information. Nothing is real until you say it at the game table. You planned for the guards to have truesight, but maybe it's better that they don't. This one feels weird when you're doing it, but I have found it to be an invaluable tool.
Involve the party
When a character is about to do something crazy, it's important to remember that you control the flow of time. Events do not have to transpire right when a player says they do.
DM: (addressing the other party members) You see your halfling friend hide from the two royal guards, and then go invisible. What would you like to do?
Bard: I'm going to walk up to one of the guards and distract them with conversation.
Fighter: I'm going to watch and see how this plays out.
Paladin: Whatever you three are doing, I don't want any part in it. I walk away from the throne room.
Pausing the flow of time gives the other players a chance to involve themselves, even if it just means leaving before everything blows up. If the other players are on board, you've turned "that one player is running off again" into "cool impromptu heist".
If you've clarified the situation enough and multiple players are discussing their options, it may be time for you to leave the room. (Thanks to Skrrp in the comments for this one.) I'll often take a break when a plan has potential, but still needs some ironing out. Like it or not, Dungeon Master is a position of authority. With you out of the picture, the only people to say "no" are the other players, and sometimes it's easier to accept "no" from a peer. Your return can also mark a logical moment to bring the discussion to a close. "So, what's the plan?"
Step through the plan's execution
Getting on the same page when it comes to a plan's details is difficult, so I've found that it's easier to take it one step at-a-time. After a long-winded plan explanation, I often find myself telling the player, "ok, so what action are you taking first?" If the player is just throwing ideas out, they'll often back down with a "well, I was just thinking". If so, then problem solved. A crazy plan is only crazy if you execute it.
So you've clarified things, involved a party member or two, and the doomed plan is still being executed. Now what? Is it time to tear up some character sheets?
In my experience, most seemingly-deadly situations can be made less deadly with a few tweaks. It's easy to say "the guards spot you trying to sneak into the throne room and you die in a hail of crossbow bolts", but that's not very fun. My go-to solution is to have the plan fail at the earliest (or safest) opportunity.
DM: (addressing the rogue) You see your bard friend chatting up one of the guards, as the paladin strides away. What would you like to do?
Rogue: I sneak past the two guards, being careful not to bump into them.
DM: As soon as you step out from behind the fighter, one of the guards points his heavy crossbow directly at you. "What do you think you're doing, small one?" He appears to be able to see you despite your cloak. What do you do?
If the player was caught stuffing important documents into their pack, it would be much harder to justify letting them off easy. Instead, the plan failed at a point where the party has an out. They may come up with a creative lie, the bard may charm a guard or two, or the fighter may bribe the guards and apologize.
There should be some consequence for failing, but that consequence shouldn't always (or usually) be death. Whenever possible, the consequence should move the story forward, rather than grind it to a halt. (This concept of failing forward is used in Dungeon World, a more narrative system, but applies to 5e as well.)
None of these steps involve telling a player "no". If I have clarified the situation, involved other party members, and the player has reached a fatal step in the plan's execution, I still don't say "no". Instead, I say, "Alright. Describe for me how your character dies." (I ask this right before the point of no return. If a player backs down here, I allow them to do so.)
That doesn't mean that you should never say no, but that you should reserve it for special cases. If a player says, "my character kills the annoying child", and your group has an (explicit or implicit) agreement not to harm children, then "no" is the appropriate response.
My answer thus far has assumed a player acting in good faith. Stepping through the plan's execution can help weed out the trolls ("I run at the guard that's pointing a crossbow at me and stab him in the face"), but it's hard to be sure. If a player is intentionally breaking the game, it's time for an out-of-character conversation.