It's such a shame that 5E is confusing in this matter. I really hope we get some Planescape/Spelljamer setting

Refering to the other answer, both spectres and ghosts have explicitly written how exactly they can phase through solid objects, the Mage Hand cantrip doesn't. So, RAW, by omission, I would say it can't do it

What I was trying to state was, the hand must be physical to interact with the objects and it doesn't say anywhere that you can't choose to "deactivate" that so it can now phase through them. Therefore assuming it is always physical