Most of the "fluffy" proficiencies are strictly terrible from a gamist perspective.
Writing set proficiency? The only real payoff I can see that having is selling your autobiography after your adventures end -- hardly necessary with the vast amount of treasure that litters the Forgotten Realms landscape. What about musical instrument proficiency when you're not a Bard? These things all have the same applications you've listed for the gaming set.
Shrewd planners force the opponent to play on their terms.
The idea of using the gaming set to win a McGuffin or settle serious conflict is perhaps less improbable than you might think. With good people skills, a character could appeal to an NPC's competitive nature and egg them on to playing a game to settle a conflict, much in the same fashion that pool hustling works. Thus, the gaming set reaches its "full potential" when used in tandem with social skills.
Campaigns of pure intrigue are also quite common.
Campaigns with very little combat, taking place in the refined halls of nobility, would invite much more usage out of the gaming set as a means of settling conflict. Sophistication appeals to the noble pallette, and a war or strategy game, such as chess, with complex rules allows nobles to exercise their mind without putting their estate at risk by engaging in actual war. Embarrassing a well-known duke by beating him at his favorite game goes a long way towards making a name for one's self.
And, of course, there's always a little cold, hard cash.
Many people in real life make a living by routinely beating others at poker, or the dealer at blackjack. Gambling games which don't rely entirely on luck have appeal to everyone; the professional gets to keep making money, and the average player gets to hope that they'll eventually be as good as the professional. Provided the settlement the party is in has a few gentlemen or ladies willing to put their money where their mouth is, it can be a good way to make a quick buck -- moreso than a simple Bardic performance if the player is lucky.
I'm not trying to say that the gaming set is the "best" fluffy proficiency, or anything equally ridiculous. The gaming set is one of those things that will be useful in some campaigns, and not that useful in others. It's the same way with the other fluffy proficiencies. The designers placed it there to encourage the players who received it in their background to think outside of the box. Each background has something like this, or something that can be used to create good character background or even a plot hook.