Yes, you can learn a demon's name this way.
Dominate monster could be used to compel a demon to reveal its true name, but the demon gets a saving throw (and most demons have Magic Resistance, so they will have advantage on the saving throw). Since revealing its true name is "obviously harmful" to the demon, suggestion would not be able to compel it in that way.
The most common use for a demon's true name is as part of the 4th-level summon greater demon spell (in Xanathar's Guide to Everything):
At the end of each of the demon’s turns, it makes a Charisma saving throw. The demon has disadvantage on this saving throw if you say its true name. On a failed save, the demon continues to obey you.
Gate can also be used to summon a specific demon whose true name is known — perhaps one too powerful to be summoned and controlled via summon greater demon:
When you cast this spell, you can speak the name
of a specific creature (a pseudonym, title, or nickname doesn’t work). If that creature is on a plane other than the one you are on, the portal opens in the named creature’s immediate vicinity and draws the creature through it to the nearest unoccupied space on your side of the portal. You gain no special power over the creature, and it is free to act as the DM deems appropriate. It might leave, attack you, or help you.
Note that in the case of gate, knowing the demon's true name doesn't help you control it; likewise other spells like planar binding or, indeed, dominate monster are not more effective if the demon's true name is known.
This answer has been edited to include new information from Xanathar's Guide to Everything, which includes spells that specifically benefit from knowing a demon's true name.