You should really request to be involved in any secret whispering happening between players. If you let them talk together then that means you are missing information, which might be relevant to you. Imagine, for example, someone is eaves-dropping on them and speaks Halfling... he'd have to know what the players said, but you don't know and asking them for a recap or saying "I want to be listening this time" would be really suspicious.
If you're simply there by default, you'll get any information you might need for your story and nobody will be suspicious.
Also look at the type of game. Players keeping secrets from one another is not the usual D&D approach, since the default assumption is that they're a group of characters who trust each other. Unless your game has in-party distrust as a key component, it might cost more than it gives you back. A lot of the time you'll be waiting for a secret conversation to play out, only to end it with "Okay, I'll recap everything to the group now", which is just dragging things out.
And finally, if you do have secrecy high up and are playing by chat and are willing to do some searching, you can add an extra level of cool by looking for an online tool that translates text, with a secret key to convert it back. Then, you can share different keys for different languages and communicate in the main channel. Only people who have the language key can read the text, but the non-verbal parts can still be done in the common language. It gives a slight overhead because of the added copy-paste steps, but it means that people who are present but don't understand the language still have something to do (as players) and still get the non-verbal clues that you would normally get when watching people talk in a language you don't understand.
You could get lines like:
John looks at his Dwarven companion for a while, then looks back to the two Elves. He speaks in his accented Elven. "Agfdkd fhdjkfg artuih sdfouidfg weriuhwer afwitre?"
If you enjoy secrecy and in-character dialogue, the guesswork this generates could be a lot of fun for the players.