Compare Blindsight and Truesight to the rules for Faerie Fire. Creatures caught in Faerie Fire are outlined by the spell. But Faerie Fire doesn't just let you sense them, it specifies that the creature does not benefit from invisibility.
Now compare the text for blur, which gives disadvantage on attacks based on visual stimuli. An attacker is immune if it doesn't rely on sight, such as blindsight, or can see through illusions, such as truesight. It spells out that its benefits are nullified.
From a rule-based standpoint, there is some benefit that invisibility has that blur doesn't, and there's some benefit to Faerie Fire that Blindsight and Truesight lack.
From a narrative standpoint, Blindsight allows you to sense an invisible creature, but not perfectly. You may be able to sense the creature is up next to you, and maybe you might be able to work out that they are attacking. But no matter how good your ears or nose are, you wouldn't be able to track finer details like eye movement or shoulder position, so you can't be sure you know how the details of how the creature is attacking or defending.
And Truesight may have similar narrative limitations. Truesight is a way to see through illusions and invisibility, but it doesn't grant infallible vision enough to nullify these effects. While it's not spelled out WHY it doesn't fully nullify invisibility, there's a variety of explanations. The spell/item that granted it might be flawed. There might be too much information for the brain to process to see through invisibility perfectly. Because of that, there's a visual delay or motion blur or slowed reactions.
But in the end, it does come down to the DM. Me personally, I'd say the blindsight/ truesight person could use their reaction to try to perceive an oncoming attack, or bonus action to try and identify how the person is defending. Sight person would use a perception check against a stealth check from invisible person. If perception wins, I'd say the invisible person was making enough noise or shifting enough dirt or telegraphing their moves too much and they gave away their defensive stance or attack pattern, nullifying the additional benefit of invisibility.