There would be "altered play" but not necessarily balance issues.
With most rules, they are written in the best possible fashion by the developers in order to promote balance and fairness. However, it is even indicated within the Player's Handbook and Dungeon Master's Guide that these rules should be a guideline, with ample opportunity to tweak them to any way you deem fit.
Since this rule would apply across ALL beings in your campaign(s), it is not exactly a "balance" issue, seeing as all creatures are affected by this equally, therefore preserving balance. It would, however, create some crazy, and perhaps convoluted fighting scenarios, though you aren't being forced to utilize your reaction in the form of an opportunity attack in the first place, again, mitigating the impact this house-rule would have on the overall flow of battle/the campaign.
As far as affecting certain features, spells, etc., most of these things are worded in a way where your attacks are still limited to specific targets. If you wish to then extend this house rule to these situations, you may experience some strange circumstances, but again, it would all be still be balanced since enemies would undergo the same changes.
A creature can be hostile without being an enemy.
While not a direct answer to your question, as it is all up to interpretation, type of campaign you are running, how combat heavy it is, along with many other things, you can attack a friendly or neutral PC or NPC with an opportunity attack under the correct (and very simple) circumstances.
In my own form of RAI:
The difference between Hostile and Enemy is that an enemy is counter to your alignment, goals, or other way of virtue that inhibits it from being your ally or even neutral toward you. A hostile creature (or other being) is something that means to do you harm, which can definitely be a friendly who is intending to cause you harm (even if in a playful manner).
However, RAW indicates for "Hostile" (DMG Pg. 244):
A hostile creature opposes the adventurers and their goals but doesn't
necessarily attack them on sight. For example, a condescending noble
might wish to see a group of upstart adventurers fail so as to keep
them from becoming rivals for the king's attention, thwarting them
with slander and scheming rather than direct threats and violence. The
adventurers need to succeed on one or more challenging Charisma checks
to convince a hostile creature to do anything on their behalf. That
said, a hostile creature might be so ill disposed toward the party that
no Charisma check can improve its attitude, in which case any attempt
to sway it through diplomacy fails automatically.
Unfortunately, there is no exact definition on what makes an "enemy" an "enemy", so feel free to adopt my own interpretation from above or develop your own beyond what is indicated in my answer.