First off: Why a magical solution, if a mundane is as good as any? Take a duffle bag. (If someone wants to search it, the good old "These aren't the droids you're looking for" is still an option.) Or put it inside a cane. People would expect you to draw a sword out of a cane if you would have to fight with it.
Of course the cane method would require your character to have a cane with him what doesn't necessarily suite any person.
The magic way is also easy: Correspondence 2/Matter 1. You connect to the sword (hence Matter 1) and draw it from anywhere (Correspondence 2). So you could draw your sword from a drawer, a weapon rack, what have you.
But remember: You shouldn't draw a katana from your pocket but rather out of a coat (or a ... duffle bag).
Other magical ways? Yes: Matter 2/3 (I'm unsure with this one. If the sword is large enough to warrant 3, Correspondence should also be 3). You take a wooden sword with you and if you have to fight you "unsheathe" your sword. (Or your mentioned pocket knife.)
Mind 3 could be used to make everyone to not look at your sword. But that would be quite the feat and if I was your GM I would demand that you take everyone you encounter/meet or even just pass by into consideration and have to either make it affect the whole city/country your in or you have to weave every passerby into it (which would be demanding).
Forces 2 could make it invisible, but not immune/hard to touch. That would be Matter since you would "change" the material component of your sword.
A spirit realm would be possible, but as you wrote yourself, it would be quite excessive.
That are about all I could think of at the top of my hat. The problem with these solutions is that it a) has to fit your mages paradigm and character (see cane) and b) you have to think about the consensus.
I would rather go with the mundane solutions since every magic contains the possiblity of paradox. And if you don't play a High Magic (so to speak) game of Mage there can be quite the problems with paradox and consensus.
Hope this helps you out a bit.