There is no such a thing as "surprise round" in 5e.
What you get is that everyone has an initiative order, and then some "skip" their first turn because they are surprised by the combat happening.
In your case:
The goblins aren't surprised by the rogue, as they have spotted before he could get the drop on them
The rogue isn't surprised, as he was approaching an enemy he saw
The rest of the party is (arguably) not surprised (more to follow).
Now things get tricky: The goblins are already engaged on the badger, so even if they are hostile to the party, they may decide not to attack the rogue. If they do decide to attack the rogue immediately, your DM may decide that the sneaking failure wasn't something the rest of the party expected, and make the rest of the party surprised and miss their first turn.
Personally, I'd just make the whole party (rogue included) roll for initiative, as they enter a 3-way fight, and then not having anyone surprised.
This is because goblins are hostile. If they were walking by, say, two knights dueling each other who didn't care, that wouldn't trigger combat for the party (unless they attacked them). If in your universe goblins aren't hostile to the party, YMMV.