So, your first mistake was allowing the players to go to Fantasy Costco and freely spend their wealth on magic items. Magical items aren't purchasable by default; the only things players can buy are what you say they can buy. The Dungeon Master's Guide discusses this on p.135.
But it's done, so what can you do to fix it?
1. They might be cheating (intentionally or not).
My first thought is that there are some kind of shenanigans going on here. Even having a lot of items should really only increase the characters' capabilities so much because of the way bounded accuracy and attunement (DMG p.136) work. If the items are allowing the party to punch way above their weight class, it makes me suspect something isn't quite right on the "following the rules" side.
Look carefully at each item and make sure they aren't exceeding the limits on how many attuned items they can use at once. That's the single largest balancing factor that prevents magic item abuse. To wit, each character can have only three items attuned, and only one of any specific item. (This rule is somewhat tucked away in a specific paragraph of the DMG, so sometimes players don't realize "requires attunement" has some specific limitations beyond being unable to easily hand the item around to other PCs.)
Make sure the players are accurately tracking how many charges their items are using and recovering. Sure, a staff of frost might let you use cone of cold long before you should have access to it, but that also spends 5 of its 10 charges, so two shots of that and it's dead, and it only regains 1d6+4 per day. Plus it's attuned, so see previous point on that front.
2. Wait it out.
If they're hitting harder than expected, it might not actually be a big deal. Increase the difficulty of their fights a little -- add a few extra monsters to each fight, or use slightly tougher monsters. They'll gain more XP for each fight, which means they'll level up faster, and they'll soon be back on the right level for the amount of stuff they have. The problem will rapidly become less of an issue. So your best bet might just be accepting that they'll be unusually tough for their level for a while, and just adjust for it.
3. Talk it out.
If you can't wait for them to just naturally even out, maybe the best thing is just an honest discussion with the players. "Hey, I made a mistake when I let you buy all that stuff, and it's messing up the game. We need to fix this, but I don't want to impose a fix on you. So can we come up with a story together that removes the strongest items from your characters?" You can back up that it was a legitimate mistake on your part by pointing out that the DMG specifically says all magic item purchases are up to the DM to allow. You didn't actively stop them from buying those items, but you weren't really aware of what you were allowing and that you had the means to put on the brakes, either.
Maybe your players will resist losing their power fantasy, but in most groups the players are more than willing to work with you on something like that, and may come up with a much more interesting story than anything you could've made on your own. (And I find that surprising the players with story twists is overrated anyway.)
Or, depending on the group, they might be willing to just hand-wave it and downgrade or remove some of the items without actually acknowledging that they did so in-character. That +2 sword was actually a +1 sword all along, the Wand of Lightning Bolts was a Wand of Magic Missile from the start, and Cloak of Protection? What Cloak of Protection? This is just my usual traveling cloak. (You can negotiate whether the excess gold just disappears, or if it gets credited to them so hey, they have a chest of treasure they can use for buying stuff later, or they retroactively acquire a small barony or something.)
4. Hit them in the dump stat.
I don't mean this literally, but rather, aim for where the characters will be weak. This is an adjunct to #2, really, but I want to call it out in specific as a strategy apart from just 'make the encounters harder'. If you can't or won't remove the items and they're playing legally, then upping the challenge is your main response, but you can also look towards what kinds of challenges you use. If they're getting a big power boost from magic items, it probably means their damage output is higher than expected, but their HP and saves won't have changed much from the usual for their level, so that's probably their most vulnerable point.
The characters are very likely to be 'glass cannons' -- that is, they hit hard, but can't take much damage in return. One or two tough monsters that can just soak the hits and deal out big damage will be a bigger challenge than a swarm of smaller monsters that don't hit as hard, even if they have the same encounter rating, because the total damage of a horde drops off as members get killed, while a big monster remains dangerous until it loses that last hit point.
I'm not saying you should kill the party, just that you may need to adjust encounters to focus more on big beefy enemies in this case.