Problem vs Opportunity
There are a few different ways to deal with the issue of players' loot and balancing the resulting crunch and power imbalance. A lot of it really depends on how you want to deal with the "problem" you're experiencing.
Without knowing exactly who's playing and what they're playing and what they were given, I would say you should analyze the crunch of what these loot items provide for the players. Once you know what new abilities/stat increases they receive and what that allows them to do, you know how to maneuver around it. Find a way to scale the power of NPCs and monsters to allow them to meet the challenge the PCs provide.
A Barbarian with a shiny new ax that penetrates armor might find themselves facing monsters that don't have armor, making the bonus useless. The Wizard with that smoking hot Wand of Fireball might end up facing some creatures that are immune to flames.
From a story perspective, if the players now are in possession of some uber-powerful (or at least relatively so) items, they might be at the whim of stronger NPCs who want to take the items for themselves. Or maybe the former owners have allies that want revenge. The problem of overly powerful loot is actually a great story hook if played right.
I think you shouldn't worry so much about "being a jerk" when you're a GM. Your players either respect the fact that it's a game or they don't. If your group is the sort that will call you out if their characters ever come up against any real resistance, then they're just sore losers. That being said, you shouldn't punish them for what you seem to feel like is your own mistake.
If you really find the situation eminently unworkable with the power disparity caused by the loot, I would talk to the players OOC about the mistake. I want to note that this would be a last resort for me, as I think this situation is easily twisted into a fun opportunity.
From an interest curve perspective, the highs of getting these awesome new tools MUST be followed by a low of having their new powers tested and probably still failing, or at least it being close enough that the overpowered-ness is negated or at least not as prevalent.
Personal Example:
As an aside, I actually do this sort of thing on purpose sometimes. In a DH-2e game I'm running currently, the players took down a noble and stole his plasma pistol. Nobody knows how to use it and it might end up killing them if they shoot it, but it's really powerful. It also only had the partially used clip remaining, further limiting the usefulness. The power imbalance potential fits well within the grimdark setting, and gives the whole group a sort of trump card.
I get it's a different setting/system/aesthetic, but the fact remains, the players could experience a number of problems from their actions beyond even just the mechanical problems the plasma pistol represents. The noble's family might come after them for revenge, or the glow of the gun might give away the player while hiding, or, or... the possibilities are as unlimited as your imagination.