What is the difference between "hostile" and "enemy"?
Why do I think there should be a difference? Well first off, they are different words. In dnd there is difference between attack and Attack action, so it's not unusual that small differences in words used are intentional. This alone hints they probably used the words on purpose where they used them.
Also I use them differently in daily use. I might act hostile towards someone that told a joke I don't like, but that doesn't make them my enemy. But of course daily life is not combat for life or death so that is a weak example.
Last, there are some game situations where in my opinion a difference makes sense. Let's assume the following situation:
We have characters A and B from the same party and C is a character hostile towards them, so an enemy of A and B. For now the words are synonymous. Now C casts "Enemies Abound" on B and B fails the save. He is now "regarding all creatures [he] can see as enemies until the spell ends.". Let's assume A hasn't noticed the spell. If A would now for whatever reason try to cast "Friends" on B, the spell should fail because B regards A as an enemy and is hostile towards him. Friends requires the target to not be hostile. However here A does not (yet) regard B as an enemy. So hostile and enemy are not the same. Contrary A is not hostile towards B so B could cast Friends on A, even though he considers A an enemy.
This makes sense to me so far. It is not limited to the spell "enemies abound", I think the same logic would apply if B was a hidden enemy in the party. Then B should be able to cast friends on A because A trusts him, but A should be not able to cast it on B because B is actually hostile towards him.
So when A interacts with B, whether B is an enemy is something A decides, however whether B is hostile towards A is something B decides. All good? No! This definition has other problems.
On PHB page 195 under opportunity attack it states:
You can make an opportunity attack when a hostile creature that you can see moves out of your range.
So in the example above B would not be able to make an opportunity attack against A because A is not hostile towards him. That makes no sense at all. B considers A an enemy and should totally be able to make an opportunity attack.
Also B could not target A with any spells that require A to be hostile.
Another example with opportunity attack:
An enemy courier has an important message in the bags of his horse. I push him off his horse, but the horse runs on on its turn (probably afraid). Can I make an attack of opportunity against the horse? The horse is probably not hostile towards me. So I can't? That doesn't make sense.
I hope I made clear why I am confused about the usage of "hostile" vs. "enemy". Maybe someone can explain me why they make this differentiation and what the differences are.