Suppose I cast Armor of Agathys at level 2 as a Warlock, which will provide +10 temporary hit points. If an attacker hits with a melee attack and deals 11 damage, does the attacker take the 10 damage from AoA? Or does AoA end before that happens because the THP are lost?
The key to this is the wording of the spell, along with the order of operation of an attack.
The spell says that the attacker is dealt damage "If a creature hits you with a melee attack while you have these hit points." Note that the trigger is "hits you."
The order of operation of an attack is that an attack and its target are declared, then an attack roll is made, followed by the determination of whether or not the attack hits, and then lastly, damage is rolled and then applied.
Since the determination of a hit is before the damage is dealt, the attacker would take the damage from Armor of Agathys before the damage of the attack is resolved.