#Nothing in the rules says they can't
Nothing in the rules says they can't
A character can use a component pouch or a spellcasting focus [...] in place of the components specified for a spell. Basic Rules, Casting a Spell
There is no special requirement that the pouch by magically bound or attuned or otherwise restricted to one person using it. It is just a pouch full of ingredients and there is no rules reason why you cannot share it with another spellcaster.
#Location, location, location
Location, location, location
However, do note that the pragmatics and action economy of such an arrangement is likely to be tricky.
A spellcaster must have a hand free to access a spell's material components — or to hold a spellcasting focus [...] Basic Rules, Casting a Spell
Firstly, you have to keep the pouch somewhere where you both can access it since you need to physically hold it in order to use it. If one of you goes to cast a spell with material components but can't reach the pouch, that person is going to be in a tricky spot and potentially unable to cast it.
Next, depending on how and where you store it, you might have to use your free object interaction to get to it:
You can also interact with one object or feature of the environment for free, during either your move or your action. Basic Rules, Combat
You may even have to use an action to access it depending on how the DM rules transferring the item between two characters would work in your exact scenario.
The DM might require you to use an action for any of these activities when it needs special care or when it presents an unusual obstacle. Basic Rules, Combat
But all this means is that it isn't an ideal arrangement but it is all fine according to the rules. However, I would highly recommend getting a pouch for each of you at the soonest opportunity.