Yes... but it may have limited success
There are no specifications in the unseen servant spell that state that it cannot interact with worn/held/carried objects (as exist for other spells). The only restriction is that such an interaction wouldn't be considered an attack (or grappling, as a grapple counts as an attack).
It has AC 10, 1 hit point, and a Strength of 2, and it can’t attack. If it drops to 0 hit points, the spell ends.
The only issue is that the servant has a strength score of 2 so any action it might take that could be contested (such as trying to snatch a weapon from someone's hand) would likely fail.
So a servant could flick someone's hat if it happened to be quick enough, tie shoes together if unnoticed, and flip a cloak over someone's head (this last one could even be worded as "folding clothes" which is given as an example in the spell). So long as none of these actions were considered an attack, for instance, if the cloak happened to be a Cloaker.