The Blight spell (4th level) works wonders for a party facing mold infestations of any kind-- while the larger oozes get to be creatures and thus 'merely' take (half, probably, of) 8d8 necrotic damage, non-magical plants (which includes molds; D&D runs on the medieval taxonomical system) just straight up die, no save.
The spell Speak With Plants (3rd level) is even better. Unlike most normal, non-magical, non-creature plants, brown mold does possess a method of rapid locomotion-- growth in the presence of fire. Using speak with plants you can ask the mold not to hurt you and your companions, and to come with you to fight your enemies. Using a standard torch, you can then lead it along a wall of the dungeon to fight off whatever else you run into down the line. Just make sure you don't run out of duration before you leave the dungeon, or, if you do, make sure you have a secure place to camp which the now-massive-and-no-longer-allied mold can't intrude into.
If you lack spells entirely, you'll need to improvise. As a non-creature, brown mold lacks any explicit requirements for things like air, sleep, food or hit points, but it may still need some of these things to survive. The only method that definitely doesn't work is fire damage, and the only methods that definitely do work are cold damage and effects that explicitly say they can kill/destroy noncreature plants.