Here are the rules from the PHB p.154 concerning gaming sets:

> If you are proficient with a gaming set, you can **add your proficiency bonus to ability checks you make to play** a game with that set. Each type of gaming set requires a separate proficiency.

It is not specified which ability checks you could use, but *Sleight of hand* (vs *Perception*, most likely) would probably be a good one for card games, for example.