You can empty your hand before casting
Retrieving your component pouch in order to perform the material component of the spell may require object interaction (if in a pocket), or even an action (if in your backpack). Instead, a component pouch is normally kept on your belt where you can access it easily.
You only need a free hand ready to manipulate your component pouch as part of the spell:
If a spell states that a material component is consumed by the spell, the caster must provide this component for each casting of the spell. A spellcaster must have a hand free to access a spell's material components -- or to hold a spellcasting focus -- but it can be the same hand that he or she uses to perform somatic components.
When a spell describes how material components are handled or used, that handling/use takes place as part of casting the spell, it does not require a separate action.
Note that you do not have to sheath your sword unless totally necessary. You can simply drop it and keep it propped up against yourself or held with your shield arm. However, another enemy could try and grab it or kick it away, so make sure you are careful.