When it comes to DM rulings, they are ultimately always correct by the rules as intended: the intention is that, as the DM, you have the final say in these matters. So that one's easy.
It's not correct by RAW, as the Mage Hand spell does not state anywhere that it causes Disadvantage, nor do Ogres list anything about having Disadvantage because they get distracted by magic.
That leaves us with "Is this reasonable?", which is the most interesting one of the three, I think.
If we check the spells section in the PHB, there is actually one cantrip that applies Disadvantage to attack rolls. It's Vicious Mockery, which is a Bard spell. However, using it requires an Action (as it's a single target spell), only applies Disadvantage to a single attack roll, and allows the target a saving throw to avoid the effect entirely. The only thing Vicious Mockery does, that your ruling does not, is 1d4 Psychic damage. Which, ultimately, is pretty inconsequential. So granting Disadvantage to all attacks for the entire 1 minute duration is very powerful for a Cantrip.
If we look further, the only 1st level Wizard spell (I found, anyway) that can grant Disadvantage on attack rolls is Fog Cloud. But that affects everyone, including your allies. So if your idea for Mage Hand was that it would distract all enemies, that makes this Cantrip more powerful than a 1st level spell.
So I'd say you made Mage Hand into quite a powerful spell by allowing it to apply Disadvantage to all attacks by enemies for a minute, without a save. (Not that it's a big problem; but I'd be careful allowing it to do this all the time)
Fortunately, it's easily explained in the fiction. Ogres are commonly known for being about as clever as a sack of bricks, so saying that the Ogre got confused by the mage hand makes total sense in the situation. But I wouldn't turn it into a regular house rule.
(If you want a more balanced version of "distract enemies with mage hand" then I'd say using your Action to target one opponent, who gets a saving throw, and has Disadvantage on their next attack if they fail. It would be balanced; that nets you a weaker version of Vicious Mockery, which is fair considering the rest of the utility of Mage Hand.)
