This is unbalancing
Assuming a character gains enough magic items requiring them to choose between several items, you can use more magic items than you could attune to at once in addition to being able to switch them using only an action instead of two short rests.
The following items could be used at the same time, all requiring attunement:
- The armor
- Flame Tongue
- Animated Shield
- Belt of Giant Strength
- Cloak of Protection
That's 4 items requiring attunement not counting the armor.
Furthermore usually if you only use a single weapon, you'd attune to a single weapon at the same time. Let's say you're in a combat with 2 enemies, one with immunity to fire damage and vulnerability to cold damage and the other with immunity to cold damage and vulnerability to fire damage.
In this fight you can be attuned to 2 other items and attack the enemies using the damage type they're vulnerable against by swapping between Frostbrand and Flame Tongue at the cost of a single action.
Loyalty and balancing
Maybe to prevent this from completely unbalancing a campaign, make it so that you need to gain the Mcguffin's loyalty to increase the number of items you can attune to it this way?
This is nothing balance-related. The fact that you need to do some roleplaying/quests to gain the benefits of the armor doesn't mean the item is not unbalancing once you're ensured the loyalty.
Assuming there are no other effects to the armor than being able to mess with the attunement rules, you should attune to another item as long as the armor's attunement limit is 1 making the armor effectively useless (you simply replace one attunement with another one and restrict the type of item you can attune to) and otherwise it's unbalanced, see above.