No, nothing especially unbalancing about it; it’s the norm in my games and it’s never caused a problem. In 3.0e, it was even official, sort of, in that Arms & Equipment Guide allowed bracers of armor to have special armor properties, rather than only armor bonuses.1 Bracers of armor use the arms slot instead of the body slot, but it’s the same idea. Magic Item Compendium explicitly allows the bracers of armor effect to also apply to body-slot items, while simulataneously allowing that benefit to ignore the price-premium for combining magic items (so a robe could be a vest of armor and also a robe of useful items at the cost of just adding those two together, rather than one costing 50% extra). Pathfinder, which is a D&D 3.5e spin-off, also allowed special armor properties on bracers of armor.2
Races of Stone also has an exotic armor (which is a thing that Races of Stone explains) called gnomish twist cloth, which is +0 AC light armor that has no armor check penalty and no max Dexterity. It is light armor—so it causes trouble with monks etc.—but otherwise it’s an official, 3.5e source of what you are describing. (You can wear it without penalty even without Exotic Armor Proficiency, by the way—what that feat does, for gnomish twist cloth, is cause the armor bonus to AC from its enhancement bonus to count against touch attacks. Which also nicely justifies its existence in a game where clothes can get armor bonuses. Though ultimately I recommend against the feat.)
So yeah, go ahead and allow that. In most cases, there are better options (nightscale from Underdark has a +2 armor bonus to AC, while still having a +10 max Dex; spidersilk from the same source has a +3 armor bonus with +8 max Dex), but if you have a very-high-Dexterity character, yes, this is totally legit.
How official to treat Arms & Equipment Guide, particularly with respect to things that saw reprint in 3.5e, is contentious. The 3.5e version of bracers of armor does not mention special armor properties, and as the latest printing can be seen as an “update” erasing it. On the other hand, it’s really an update to the original 3.0e bracers, which didn’t mention special armor properties either—so maybe the Arms & Equipment Guide modifications to the 3.0e bracers should apply equally-well to the 3.5e bracers. Anyway, this is all semantics—it doesn’t matter what the “official” rule is so much; the real question is what’s best for your game. And allowing it is definitely best for your game.
Technically, bracers of armor—in both D&D 3.5e and Pathfinder—can have an armor bonus of up to +8, while real armor is limited to magic enhancement bonus to armor of +5 (in 3.0e, real armor apparently could also go up to +8 enhancement, which is probably where bracers of armor got its +8 cap in the first place). Pathfinder didn’t change the +8 limit when it allowed bracers of armor to get special properties, it just said the total of enhancement and special properties cannot exceed that same +8—even though the limit for real armor’s enhancement-equivalent bonuses is +10 (Arms & Equipment Guide allowed bracers of armor to go to +13 total enhancement-equivalent, but again apparently that was also the norm for real armor in 3.0e). But anyway, I have never enforced that +8 limit on bracers of armor or on clothing-enhanced-as-armor, and see no reason to do so; it can be +10-equivalent just like real armor (I’m not a huge fan of the +13 limit from 3.0e—the costs for that are so high I think it’s good for the rules to tell people not to do it). The limit on actual-enhancement-to-armor, whether +5 or +8, doesn’t matter because no should ever hit either (the minimum +1 to get started is all any armor should ever have).