What is 'railroading', and when is it generally considered to be something negative?
Railroading is forcing the characters into the prewritten story that the master created. It's generally frowned upon, because it disrupts the free-will oriented nature of roleplaying. In some cases however, some railroading is required.
A typical example is the following. Suppose the characters enter a city, and find a riot or similar event. The most sensible choice would be to get the hell out of there immediately. If the players choose this strategy, and the master needs their participation in the riot for the plot to develop, characters will find the city door closed, or a mob in front of it which prevents them to leave.
A different type of railroading is hidden and more subtle, and players don't realize it's railroading. Suppose there is a final match with the main villain. The Master decided that, when defeated, he will not immediately die, but instead he will first deliver an essential final piece of information to the characters. Suppose this happens and, while he starts chanting off his final piece of evil speech, one of the players have his character cast "disintegrate" on the villain. According to the rules, the villain will become dust. This would ruin the epilogue of the game to everyone. A sensible Master will work around this event somehow, such as granting the villain a ring of counterspell with a disintegrate, even if it wasn't supposed to have it. Only the Master know the villain full equipment in details, and he can use this point to his advantage.
Similar techniques to the one above have the Master throw a critical die behind the screen, disregard the result and do a plot-decided outcome anyway. This gives the illusion that the outcome is random, while instead it was decided by the master since the very beginning.
"Railroading" and "sandbox" are two opposite ends of a spectrum, and as a result both are good in varying degrees.
Really, railroading is any in-play modifications the GM makes to the world to accomplish his own story or other goals. In computer gaming it's called "linear." You are going to go from set piece A to B to C, most likely in that order, your other option is pretty much just not to play. It is opposed to sandbox or open-world play, where at its extreme the GM lets players wander around in the woods for a long time; he doesn't move the adventure to meet them or otherwise guide or constrain their activity. People come up with other names for railroading, really railroading is a pejorative term attached to the activity when someone finds it jarring or limiting to their play experience.
Everyone mixes the two. When someone says "I let the PCs wander around, but if I prepared a dungeon over to the east and they go west, then I move it to the east" - that's a mix of the two approaches. "Too much sandbox" can be boring and directionless, but "too much railroad" can be lame and constraining.
Now, "railroading" is mainly an activity done to further a story or narrative goal. Some games (not D&D, but there are a lot of games that aren't D&D) have this as a specific feature of their play. The entire RPG (many storygames are like this) are structured to be railroad-only. The Shab-Al-Hiri Roach is a good example. Many traditional games implicitly have the "free will oriented" roleplaying Stefano mentions but there are many games that specifically do not.
Many megabytes have been written about balancing the two, giving the "illusion of free will" to the players while really guiding them to try to balance sim and narration, whether "fudging die rolls" is good to balance game and narration, etc.
It originates in the concept of rail travel: it's on tracks, you can only go forward or back.
The simplest example of a railroad plot is a dungeon of one way doors that PC's can't jam open nor blast through, or the infamous "Teleport Dungeons" of both Gygax and St. Andre... You are dumped in a room, have an encounter, survive it or not, find the exit, repeat until exit or death. No choices are present in the more severe forms, as if rolling along a set of rails like a Disney ride.
The term is also used for less linear plotlines, especially when player choices are either very minimal or have little overall impact, or all choices eventually converge on a single outcome.
In small amounts, railroading can be useful. In large doses, players come to realize they don't have any plot-impact, and often then to resent it.
The antonym is generally considered "Sandbox," where at its extreme, the GM has no plot whatsoever, just has a setting and handles the reactions of the NPC's and Monsters to the players without any real plot for them to discover.
For the majority of players, somewhere between these two extremes is the ideal; a plot to be discovered, impact on the setting, and meaningful choices are all valued by most players.
In my experience, the amount of sandboxing or railroading can be contingent on how long your gaming session happens to be.
If I am running a full long session (say about 6 hours) then I think that the players prefer lots of sandboxing. I like to just put out the map and say, "Go anywhere... do anything."
If I am running a shorter session (say about 3 hours) then I do my sandboxing by bookending it. What I mean by that is that I will sandbox for a few minutes at the start of the game. Perhaps the PCs wish to visit a part of the town that they have yet to experience, or maybe they wish to go see that interesting NPC from a few sessions ago. Once they have sandboxed a while, I will then try to insert the prime adventure hook. Once they take that hook... then the prepared adventure can begin. Since the session is short, there will be a little railroading in that the number of choices will be more limited than normal. The PCs may only have a couple of choices on the path to take, or perhaps the order in which the prepared destinations are visited will be up to them, but all in all a prepared plot will ensue. Of course the outcome of the story will still be up to how the PCs play. After the prepared adventure is completed, I will then open up the game for sandboxing again at the end. Perhaps they want to act on information revealed in the adventure, or just wish explore the world some more. It's up to them.
So I find that if you sandbox exclusively at the expense of all else... the PCs will often enjoy the experience, but they will long for a good hook to engross them into a story. That being said, if the adventure is all railroad, the PCs will feel useless to the story and feel that their decisions are without impact or consequence.
A good GM will have a feel for this sort of balance and temper it according to session length, the game being played, and the taste of his players.
To my mind, railroading means taking any and all meaningful choices out of the hands of the players.
It is not a style of play that I enjoy but some folks dig it.
Depends on the gaming group. Some groups want and need a clear Focus, lest they stall and dither, and a 'standard published adventure' (in most cases providing One specific goal or setting) -- so termed for neutrality; 'railroad' has negative connotations -- is what they're accustomed to and what they need. Other groups may prefer an array of options, requiring the GM to have several 'standard adventures' (or equivalents in rough-outline form) at hand.
And of course, a GM may become aware of a group's needs before the group does. ;>
I am not going to repeat the railroading vs sandbox argument above, however I submit that most games have some soft, positive railroading - the "read the pink box" section that begins any published adventure.
GMs set expectations of the kind of play, because they've prepared the social event according to what they expect the players to do.
One way it begins to become negative is when the GM prepares a railroading and then gives the players a mixed message by injecting some play that suggests the players aren't stuck with it. If the players play it, they will carry away negative feelings if it is clear they never could have changed the outcome.
One big trap to avoid (roll d20 to save!) is powerlessness vs helplessness in games such as Call of Cthulhu. Characters may become helpless in certain situations, but players who feel powerless to control their character actions will feel railroaded and generally unhappy.
Railroading is fine, but I think it's best if you don't try to trick your players. Just be honest with them: "Hey, I spent a ton of time prepping this adventure and I think you'll enjoy it. So have fun & explore, but forgive me if I have to say something stupid or illogical to keep things on track."
We all enjoyed Half Life 2, and it's a completely railroaded/scripted adventure. A good railroaded adventure can be just as fun. On the other hand, there are games like Skyrim or Fallout, which are more open and exploratory, but you will inevitably bump into the invisible barriers in the game.
Personally, I prefer more of the independent rpgs, because a lot of their design went into solving the railroad problem, and have a nice secondary benefit of not requiring hours and hours of preparation to play.
My favorite quote from Apocalypse World & Dungeon World: Play to find out what happens. If everything is pre-scripted, you miss out on experiencing that.
As always, just my two cents! :)
As others have mentioned, "railroading" means taking normal, reasonable options out of the players' hands, and forcing them down a specific choice/path - like being stuck on a railroad.
There's two ways this can go, one is good, the other is bad.
The players know that this is how the game is going to go, and have no problem letting the GM make these decisions for them. This is how a lot of mass scale "organized play" campaigns work - they have a linear set of adventures, you show up, you play, and you come back next week. A lot of gamist play based around combat adventures does fine by this.
The GM keeps telling the players they have full choices for their characters, but actually maneuvers/engineers/punishes their characters into taking the one or few limited options for play to continue.
The badness of this is that you are literally lying to your players and successful play is when you can lie skillfully enough they do not realize you've manipulated their choices all along. That said, the "successful" version is pretty rare, most of the time people realize they're on rails and they keep pushing at the edges trying to find ways to take control over their characters which they are not allowed to do.
The difference between the two options is whether the players wanted to play that game and whether the GM was honest about how the game works.