Backstory
##Backstory## II want to make myself a homebrew XP system to better deal with how to reward experience points to my players for roleplay, spellcasting, and combat respectively, with a separate XP track for each, rather than just the 1 experience bar that gives limited rewards as you level. I want to create a better system where the results of my player’s actions are better reflected than a basic standard number. This leads to interacting with hostile NPCs as an encounter that can earn XP.
Actual Question
##Actual Question## WhatWhat are the stages between a Hostile NPC and a Friendly NPC. I'm looking for guidelines or rules that assist the DM in determining how the players actions and rolls effect a hostile NPC.
I would prefer references to the PHB, MM or DMG.