Skirmisher allows you to move, as a reaction, after an enemy ends its turn within 5 ft of you, so that enemy will have used all its actions. The enemy does not have to have moved, where it was when it started its turn does not matter, neither does it matter where you were when it started its turn (if, for example it used some ability to move you). Also the opponent does not have to have interacted with you (it might have attacked someone else).
Remember, you only get one reaction, so if you've already used it (opportunity attack, uncanny dodge, etc.) you can't use Skirmisher.
What is of significant importance though, and you will have to discuss this with your DM, is the order that multiple enemies act in.
Your rogue is 5 ft away from 2 opponents and in line of sight of a 3rd. One of the opponents 5 ft away is incapacitated and all 3 have the same initiative.
The order that the DM resolves the opponents' movement and actions has a huge effect on how useful Skirmisher is. If the incapacitated opponent goes first, you move before being attacked at all. If the 3rd opponent goes first and makes a ranged attack you don't get to use Skirmisher to get to cover before that attack.
Obviously there are many other possibilities, but you get the idea.
I strongly suggest that some kind of house rule be implemented to avoid arguments. Ranged attacking->melee attacking->not attacking makes Skirmisher a lot less useful than not attacking->melee attacking->ranged attacking. Of course having all the opponents act in a specific order within their initiative works best, but makes combat a bit more complicated.