Here are the rules from the PHB p.154 concerning gaming sets:
If you are proficient with a gaming set, you can add your proficiency bonus to ability checks you make to play a game with that set. Each type of gaming set requires a separate proficiency.
It is not specified which ability checks you could use, but Sleight of hand (vs Perception, most likely) would probably be a good one for card games, for example.
Another example: in a bluff game (like poker), you could use Deception (vs Insight) and add your proficiency bonus if you are proficient with this particular game.