#No.
We can extrapolate an answer based on how Dispel Magic interacts with spells like Bless:
From the Sage Advice Compendium
If dispel magic targets the magical effect from bless cast by a cleric, does it remove the effect on all the targets? Dispel magic ends a spell on one target. It doesn’t end the same spell on other targets.
Since the conjure spells actually bring creatures into the Prime Material plane and in essence holds them there, a Dispel Magic will only end the magic on a single creature the rest would still have the magic to keep them present.
