No. It's not the size that matters ...
The armor class of any creature will reflect how difficult it is to hit. Where the rules identify a specific source of disadvantage (for example, a Sprite when invisible will cause disadvantage1 on an attack against it) the general case has no direct linkage between size and advantage/disadvantage in hitting such a creature.
During my last session a player mentioned that as my large monster went to attacked a tiny PC, the monster would have a disadvantage
That may have been a rule from a previous edition, but isn't a general rule in this edition. There are some creatures, however, who can benefit from cover due to their small size.
For example, the Lightfoot Halfling (SRD p. 5 / PHB) has a specific racial feature unique to that character race:
Naturally Stealthy. You can attempt to hide even when you are obscured
only by a creature that is at least one size larger than you.
Being hidden/obscured will negatively influence attack attempts against that creature. That's a case for "specific over general" making that creature more difficult to hit sometimes.
1 The sprites invisibility and the "unseen attackers and targets" rules combine in this case.
Invisibility. The sprite magically turns invisible until it attacks
or casts a spell, or until its concentration ends (as if concentrating
on a spell). Any equipment the sprite wears or carries is invisible
with it. (SRD, Sprite, description).
Unseen Attackers and Targets
Combatants often try to escape their foes’ notice by hiding, casting the invisibility spell, or lurking in darkness. When you attack a target that you can’t see, you have disadvantage on the attack roll (PHB)