I came across a situation in my current game where a PC was taking an attack of opportunity against another PC when they moved out of their threatened square. They decided to use a non-lethal attack to attack the PC moving out of the threatened square. In this case, would the PC moving out of the threatened square also get an attack of opportunity against the PC who is making an unarmed attack since be default that would cause an attack of opportunity?
You can take an unarmed strike as your attack of opportunity even if you don't have Improved Unarmed Strike, but you first have to be threatening with a weapon. If you have a dagger in one hand and your opponent leaves the threatened square, you may take an attack of opportunity using any means available to you to make an attack. A dagger is enough to threaten the square, and the text for attacks of opportunity does not state that you need to attack with the weapon you were threatening with. You can be armed and still punch (or kick, elbow, knee, etc.).
Assuming that you already threatened the square and made a legal attack of opportunity, then that will provoke an Attack of Opportunity from the opponent if you lack Improved Unarmed Strike.
Likewise, if you had used your Attack of Opportunity to try and trip the opponent without Improved Trip, that would have the same effect.