How can I make my players stop speaking out of character?

Question

I'm a GM running my own homebrewed system, in a campaign with three players.

My players rarely talk to each other about what to do, forcing me to make suggestions that basically give away plot.

The real problem is that, even when they do have discussions, two out of the three break character when it's not an acceptable context to do so. They stretch it out as much as they can. I've talked to them before about how it'd benefit everyone if they discussed more and in-character.

I have tried penalizing their health and even their gold, but it doesn't seem to send them the message. Worse, when I subtract anything from them they just laugh and say, "It was worth it."

Subtracting from their health nearly leaves them dead when combats happen, and if I penalize their gold they run short in interactions involving currency.

Should I just begin killing them off and give them new character sheets if it goes too far?

Answer 1

Address the players in Character

Address the players as the NPCs who talk to them. Look them in the eye, make the statement or question, and then hold the gaze until the answer. Even if they answer in the third person, continue in character responding to what is said. It sets the tone, gives them the example and if you're acting in character they will be more likely not feel weird about being in character.

Reward

The rules for some games allow the DM to give rewards. D&D 5e, for instance allows for XP and Inspiration. Inspiration is nice because it is immediate reward for doing the action you want to reinforce. If your game system doesn't have inspiration, you can house rule some form of immediate reward.

Treat Table Talk as in Character

One way I've seen this done is take their OOC banter/side talk as in character comments. Especially if they are discussing some course of action they think might be funny.

GM: The well dressed gentleman passes by and eyes you up and down.
Player (to other player): We should rob him. Right? It'd be funny.
GM: (as the gentleman) "Rob me? Should I call the town Guard?"

This is a trick I got from Matt Mercer in Critical Role. I noticed how it worked to refocus the players on what is going on in the game, and encouraged more roll play.

Talking Through Things in Character

Give them puzzles they have to work together to solve. If the character doesn't say something (the player does), the other character's didn't hear it. This works well for timed challenges, but could be useful in other puzzles.

Sidetracked

Set a time limit, especially in battle, on what the character wants to do. If a player is taking an unreasonable amount of time deciding or is asking too many questions he should already know the answer for, the character loses his initiative and gets moved to the end of the round. The serious players who have been paying attention will thank you. Another one you will see Matt Mercer do, but that wasn't the first place I'd seen it. The nice thing about this is that it keeps the game moving which makes it easier for everyone to stay engaged.

It's a Game

Sometime, if they aren't in battle and there isn't any threat, letting them get a little off topic can be alright. It is a game, and games are meant to be fun. I don't think the whole session should be taken over, but a little levity might just be what they prefer in the game. It could be that what they want out of the game and what you want out of the game are different, and brief talk about what everyone wants might be in order. Especially if you never had a session 0.

Answer 2

Don't make players do anything

Let players use OOC talk if that's how they want to think and plan. Just don't let it devolve into too much crosstalk ('too much' itself being a sliding bar, different for every group).

Remind your players OOC discussion still represents IC discussion, exactly the same way as saying "I swing my sword at the orc" represents swinging a sword at the orc. OOC talk may not translate verbatim, but the characters aren't just paused while OOC talk is happening. They're also having a discussion.

If they discuss things which some NPC might be in a position to overhear and react to, then interrupt with a reaction. But if they're in their hotel room alone, just let the players who prefer OOC talk to talk OOC.

Answer 3

Absolutely DO NOT punish your players via health and resources. Taking "tangible" things away from the player makes the players think your game is DM vs. Player, and they will continue to rebel against you. Not only that, when you take away in-game resources without reason, you reinforce out-of-character behavior, rather than focusing their attention on the in-game world. The resources might be in-game but those resources are often as important or more important to players as they are to their characters.

LegendaryDude has rightly questioned this type of punishment,

"It seems odd to me that in order to correct/reduce [out-of-character] activity, that punishment handed out by the GM would also be in a meta-game, [out-of-character] form".

There are better ways to encourage behaviors. We want to reinforce in-game behavior with in-game consequences. We also want to handle it in a fair and balanced way. Let's go with a Carrot and Stick approach.

The Carrot

You're punishing with the stick, but sometimes you need to lead with a carrot instead. What I mean by that is, you need to incentivize behaviors you want them to have. One way you can do this is by awarding bonus role-playing XP at the end of the session. I've had this used on me in the past (as a newish player) and it's a great way to get people engaged and interested because it encourages them to actually try to role-play with mechanical rewards. People that are only interested in rolling dice are more or less forced to be their character if they want to keep up with a player that is role playing. Additionally, this pushes the out-of-character rewards to the end of the session, so it keeps players focused in-game while still utilizing meta-game rewards.

The Stick

Obviously, the carrot doesn't always work, and you've clearly felt the need to use the stick. But, again, don't take mechanical things away from them like gold and health. If your problem is (and please clarify for me) that your characters are discussing things for too long, put them on a timer. "The merchant, annoyed and bored, walks away from this conversation". Oops, they just lost an opportunity. You can treat out-of-character discussion as in-character discussion as well and teach them to be careful of what they say. Make it obvious that when they're talking, they're talking in character and everything they say matters.

Dan Henderson also rightly notes,

The [out-of-character] talk happens because your players are not immersed in the game world. When you subtract gold or health without an in-game reason, you only contribute to that underlying problem. For a punishment to be effective at curbing [out-of-character] chatter, it has to increase immersion, not decrease it.

At the end of the day, it's important to recognize that DnD is a social gathering, and people like to make jokes and talk and it's difficult to remain in character for several hours at a time, and because of this it might be fitting to find a compromise...

Allow Out of Character Discussion Sometimes

You might not like it, but out of character discussion is very handy and helps people from becoming frazzled at playing a character for several hours straight. Allow them to discuss certain things out of character, but make sure there is an appropriate way to do it. I suggest the "hand on forehead" approach. When a player has the top of his hand pressed against his forehead, he is speaking as a player and not a character. This is useful to allow jokes, ask questions, and discuss options about what the players want to do. This can be abused, so you may need to use the stick again and put them on a timer if it goes too far. In my experience, it's incredibly useful to keep everyone on the same page. No more "wait, was that in character or out of character?". You as DM need to treat it as law, so if they make a joke without the hand on the head, they made the joke as a character, so you can react appropriately. In time, they should learn to use it effectively.

Talk to your players

This is basically the number one thing. You and your players need to be on the same page. You seem to want them to play a very immersive campaign where every player is their character. They seem to think otherwise. Before this campaign started, you should have let them know what kind of game you were going to run. Well, it's not too late to have that discussion. Let them know what type of campaign you're going to run and how you're going to run it, what you're awarding, what you're punishing for, etc etc. You're a player too, after all, and you'd like to have fun a certain way. Of course, you could always change the way the campaign is run to suit their play-style, but this is less than ideal.

Answer 4

I'd like to offer a frame challenge.

If I understand you correctly, you want to run a game in which there is little or no out-of-character communication.

Your players have told you that they want to play in a game in which there is lots of out-of-character communication.

Why are you playing in the same game together? Either you'll punish the players until they give you the game you want, or you'll fail and they'll get the game they want, but there isn't a solution that will make you all happy. (And I notice you haven't asked for a solution that will make you all happy -- you've asked for the best way to make them do what you want.)

You need to find some players who like the same sorts of RPG that you like. Your players need to find a GM that will let them play the sort of RPG that they like. The question you asked was "how can I make my players do what I want?" but I think the question you need is "how can I find some new players who will want the same thing I do?".

I don't know what area you live in, but I've found that meetup.com is a good way to find new players. You might also try your local game store.

Answer 5

Talk to your players

You want an immersive role-playing experience where out-of-character discussion is off limits. That's great ... unless your players want to treat the game as a tactical war game and couldn't give a rat's about character development. Both of these are equally valid ways of playing a role playing game.

You can't make your players do anything

Unless they are your employees or children you, as game master, have zero authority to compel people to do anything!

Compromise or quit

If you punish their characters and they "laugh and say, 'It was worth it'" then that is a pretty clear indication that they are not interested in playing the game your way.

You need to play the game their way, negotiate a position somewhere in the middle or quit as GM.

There is only one person in the world you have the power to change - you

Answer 6

"Should I just begin killing them off and give them new character sheets if it goes too far?"

Never. That's counter-productive: the role-playing you seek requires the players to be invested in their characters, and there's no way that's going to happen if you kill them off because you don't like what they did. Learned that lesson the hard way myself when I first started DMing.

Let's take a step back.

One thing to keep in mind is that not all OOC talk has to be bad. Just as there are skill/ability rolls for things the character can do/know but the player doesn't, it's possible that a character might be clever enough or knowledgeable enough to figure out tactics or strategies that its player--or even the DM, via a skill check--might not be able to come up with on their own, and multi-person discussion may be necessary to accurately represent a character's intellectual prowess. This is even more true for new/forgetful players, who would likely benefit from OOC reminders of mechanics that don't make sense to talk about in-character.

That said, it sounds like your problem may go further than that. Your note that the "players rarely talk to each other about what to do, forcing me to make suggestions that basically give away plot" sounds to me like one of two things is going on here: 1) players may not have a good idea of what motivates their characters, so they don't know what to try to do. 2) players have a good idea of what motivates their characters, but feel like you haven't provided them the opportunity to try to act on those motivations.

In the case of (1), as others have suggested, it's possible that they don't know what motivates their characters simply because they don't care for the role-playing aspect of the game, in which I agree with the advice to find a more suitable player/DM match-up.

Normally, the fact that you've "talked to them before about how it'd benefit everyone if they discussed more and in-character", and they still didn't make an effort, would make me think that is what's going on here, but the fact that you've punished them for not playing the way you want gives me pause; it's also possible they haven't bothered figuring out what their characters want because they think you're just going to take it from them the next time you don't like what they did. If that's the case, there's still the opportunity for a fun campaign with the current people, but it will require some mutual forgiveness and understanding.

In the case of (2), the players may not be acting in character because they don't feel in control of what their characters are doing anyway. In this case, punishments just make them feel even less in control, and possibly the OOC discussions are they only thing they feel they do have control over, which could explain their "worth it" comment. Drop the punishments, figure out character motivations, describe a world where those motivations are actionable, and let the players make their own creative choices.

Answer 7

J. A. Streich nailed my solution but since it's buried in with several others I'm going to separate it out into it's own post:

There is no such thing as out of character talk. Anything the player says that's game-related but not actions or requests for more detail (or answers provided by other players to requests for detail) is something their character says and at the voice level they used to say it. (You argue in a loud voice, your character is also speaking in a loud voice and is thus easier for the monsters to hear.)

You plan that attack on the monsters and they'll probably hear you. If they're intelligent and can understand the language you're speaking they're likely to use the information against you.

Answer 8

There are only two situations where it is okay to punish players for OOC chat.

1. If one or more players are rudely interrupting another player trying to speak who has the floor.
2. If it is becoming such a detriment as to make the game no longer fun.

For the first situation, my solution is to play this video on a loop at whatever volume is necessary to drown out the offending players until they get the hint, then I stop the video and warn them that while I don't mind them chatting out of character if they persist in rudely interrupting other players and hogging the spotlight I will kick them out. It's worth mentioning that I've actually only had to do this once, because I play with polite people who aren't complete and total jerks.

The second situation is more difficult to deal with because what might make the game unfun for some people might make the game more fun for others. I generally put the issue to a group vote: if the majority of players are not having fun, then and only then do I intervene. If the offending player(s) continue, they get kicked. However, it sounds to me based on your question that you are the only one whose experience is being harmed by the chatter. If that's the case, then I hate to be the bearer of bad news, but the only solution is to find a different group. If you persist in punishing the other players for doing what makes the game fun for them, then eventually they will get tired of it and leave and you will have to find a new group anyway. Better to end things sooner rather than later.