The Worg
Although the hobgoblin and the worg are both a CR 1/2, their CR calculations are vastly different and the hobgoblin is wayyyyy more dangerous when they have their worg by virtue of their Martial Advantage feature which lets them deal an extra 2d6 damage when an ally of theirs is within 5' of their target. And that's what the worg is. Furthermore, the worg changes the hobgoblin's speed to 50, has more than twice as many hit points, has a lower armor class (and is thus an easier target), and has base attack damage that's much higher than the hobgoblin's (barring Martial Advantage).
Using the Monster Statistics by Challenge Rating table from page 274 in the DMG reveals the following:
Worg CR Evaluation
When reviewing the worg's CR calculation you'll find that at 26 hit points and an AC13, the Defensive CR is a paltry 1/8; however, with a +5 to hit and an average damage per round of 10 plus a chance to knock you prone, you're knocking on Offensive CR that's somewhere between 1 and 4, maybe a net of Offensive CR2. If you banish the hobgoblin, the worg is still dangerous on it's own and given that many times in D&D the best defense is more offense, this is a threat you need to eliminate ASAP.
Hobgoblin CR Evaluation
So let's take a look at the hobgoblin's CR more closely. If you banish the worg, you take away the strategic advantage that hobgoblins have built into their CR calculation and thus make them much less dangerous. With 11 hit points, they'd sit at the Defensive CR of 1/8 but their high AC of 18 isn't seen until CR13; obviously we don't take the average on this but you get the idea they've minimal hit points under all that armor. Their Offensive CR component isn't that impressive either, they barely tip over the 1/4 mark by themselves with their Longsword or Longbow which average only 5 damage per round. However, the Offensive CR goes all the way to 1 when they get to regularly add in their Martial Advantage feature.
If you just eliminate the hobgoblin, you still have to deal with 26 hit points barreling at you at 50'-100' per turn. And once it gets there, it's going to deal a lot of damage plus be knocking allies prone.
If you eliminate the worg, though, you just have a hobgoblin running at you firing their longbow. Given that most parties have ranged options to return fire with, this is definitely better. Scratch that, it'd have to burn an action to drop its shield (Getting Into and Out of Armor, PHB 146) to fire the longbow, which just makes your situation all the better.