Intrument of the Bards only works on spells that impose the charmed condition
When you use this instrument to cast a spell that causes targets to become charmed on a failed save, the targets have disadvantage on the saving throw.
Spells that don't say that they impose the charmed condition do not count as imposing that condition and thus do not meet the criteria in Instrument of the Bards.
For example, otto’s irresistible dance does say:
creatures that can’t be charmed are immune to this spell
However, it says nothing about imposing the charmed condition which is the only thing that matters for the effects of Instrument of the Bards.
Jeremy Crawford also supports this here:
"Charm spell" has no meaning in the rules. A spell refers to the charmed condition if that condition is relevant.
and also here:
When the game refers to being charmed, it's referring to the charmed condition. There isn't a general category of charm effects.
Side note: RAW/RAI the Instrument of the Bards' effects can only be used on spells with material components
There is another reason why otto’s irresistible dance would not work with Instrument of the Bards as well.
Since Instrument of the Bard says:
When you use this instrument to cast a spell
This means in order for the Instrument of the Bards' effects to apply to a spell you cast, it must be used as a spellcasting focus (or be the source for the spell). And spellcasting focuses are only used when a spell requires material components.1
Jeremy Crawford also backs this up:
Q: If attuned to an Instrument of the Bards and cast Charm Person, does the target make its save with disadvantage?
A: Charm person doesn't benefit from an instrument of the bards, since the spell has no material component.
1 - It also means that it can only be used on bard spells (see Does using Instruments of the bard gives disadvantage to enemies if I use it to cast wizard spells?).