That only matters if a series of things are false. If any of them are true, you're off the hook.
Either a) it didn't change the game...
If it wasn't written down, it didn't happen - Ryan's Law
First, if the baddie has only been collecting information so far, but hasn't acted on it, then this is simplicity itself: The baddie loses that knowledge. Since Bad is an NPC entirely in your head or notes, that's easy.
The same applies if Bad has acted on it, but the action failed, the party weathered it, or didn't have a significant effect on the game.
Or b) it could be attibuted to luck...
Once is happenstance. Twice is coincidence. Three times is enemy action. - military proverb
Maybe if we set aside the scrying altogether, Big Bad is being ordinarily active, sending forces out to seek pure chance encounters. Or maybe Bad's forces are not even seeking that hard, and the party got unlucky and stumbled onto one. That would be more likely if the party is seeking clues in particular places connected to Big Bad.
If any of your game-changing events could fit that picture, you are off the hook.
Or c) it could be attributed to other intelligence gathering
If Big Bad is that interested, surely there is other intelligence gathering being done. Perhaps one of those leads paid off, e.g. A spy at a crossroads was paid to be on the lookout and signal when the party came through.
Or d) Bad could have taken countermeasures
if Bad was aware of the robe, he may have been able to take countermeasures during scrying to avoid being seen. Maybe Bad has some NPC-only power, or just took care to scry from places and angles where the wizard was unlikely to see.
Either Bad just presumes everyone has sensors, or Bad got specific intelligence from an observer that the party had a Robe of Eyes.
These last two raise a sobering consideration for the party. How does Bad know so much about the party? Could someone inside the party be leaking information? Cue paranoia...