Conceivably by lip-reading, for example with the Observant feat
The Message spell says:
You point your finger toward a creature within range and whisper a message.
The target (and only the target) hears the message and can reply in a whisper that only you can hear.
Spells do what they say they do, so this means no one else can understand the message "by listening to sound," as you asked in one part of your question.
However, the spell says you "whisper", and we are to take that verb in the ordinary dictionary sense, which is (OED):
whisper: verb, a. intransitive. To speak softly ‘under one's breath’, i.e. without the resonant tone produced by vibration of the vocal cords; to talk or converse in this way, esp. in the ear of another, for the sake of secrecy.
Whispering in no way restricts or avoids the movement of the lips, and there is at least one way a player can be assured of having the ability to read lips, which is to take the Observant feat (p.168 of PHB), which includes:
If you can see a creature's mouth while it is speaking a language you understand, you can interpret what it's saying by reading its lips.
Notice it says "speaking" which is indeed in the definition of "whisper," above; whispering does count as speaking.
Thus if you have this feat and can see the spellcaster while casting and understand their language, you can interpret the message.
To clarify about the verbal component (as pointed out by Szega), it was clarified in discussion of the suggestion spell on p. 17 of this Sage Advice Compendium that the verbal components of a spell are "mystic words," separate from any "normal speech" entailed by the spell description.
But the Message spell's having a verbal component other than the whisper itself does not matter for the purposes of your question. Casting it involves whispering, which is a kind of speaking that can be lip-read.
Naturally, the same conditions would apply for someone potentially lip-reading the recipient's response to the Message spell.