There are several ways you can use magic to help identify a doppelganger.
A first clue that not all is as it seems with the doppelganger is that because their technical creature type is monstrosity, not humanoid, they will be unaffected by spells that specify a humanoid target - such as hold person. Doppelgangers are also specifically immune to the charmed condition, so any spell that relies on the subject becoming charmed will likewise fail. However, in character it would probably be difficult to identify that the doppelganger didn't just manage to resist the spell, as is represented by a successful saving throw.
Moving on to spells that could more specifically identify the doppelganger or have special interactions with such shapechangers...
The 2nd-level spell detect thoughts allows you to read the surface thoughts of a creature without even granting a save, and is noted as being useful in interrogation:
You initially learn the surface thoughts of the creature--what is most on its mind in that moment. [...]
Questions verbally directed at the target creature naturally shape the course of its thoughts, so this spell is particularly effective as part of an interrogation.
A doppelganger's surface thoughts are quite likely to give them away, especially if they are asked leading questions at the same time or the caster successfully uses the "probe deeper" effect of the spell to glean a better understanding of the creature's mental state.
The 2nd-level druid spell moonbeam has a special interaction with shapechangers:
A shapechanger makes its saving throw with disadvantage. If it fails, it also instantly reverts to its original form and can't assume a different form until it leaves the spell's light.
Moonbeam is a damaging spell, so this is a decidedly unfriendly way to try and identify shapechangers in the midst - but if the party has a reason to suspect that they are host to a doppelganger, they could conceivably use it as a way to ferret them out, assuming they have the ability to tank or heal the damage they'd cause themselves while doing so.
The 2nd-level spell zone of truth can be used to simply force a creature to tell you if they are a doppelganger:
Until the spell ends, a creature that enters the spell's area for the first time on a turn or starts its turn there must make a Charisma saving throw. On a failed save, a creature can't speak a deliberate lie while in the radius. You know whether each creature succeeds or fails on its saving throw.
The caster knows whether or not a creature is affected, and even if they succeed they must save again every six seconds (of the spell's 10-minute duration) so they are effectively guaranteed to fail eventually - at which point the caster can simply ask them "are you a doppelganger?" Unless they can directly say "no", you've found the impostor. (Affected creatures don't have to answer and can answer misleadingly by saying things which are technically true, but there's no way to weasel around a direct yes or no question of that kind, and you can take silence or refusal to give a straight answer as an admission of guilt in the circumstances.)
The 3rd-level spell glyph of warding could create a ward which only activates in the presence of monstrosities, or more specifically shapechangers, or even more specifically just doppelgangers:
You can further refine the trigger so the spell activates only under certain circumstances or according to physical characteristics (such as height or weight), creature kind (for example, the ward could be set to affect aberrations or drow), or alignment.
A glyph of warding that goes off only when activated by a doppelganger would positively identify such an impostor - either explosively if used in the explosive runes mode, or less violently if it simply casts some sort of identifying spell in response to the doppelganger's presence (major image to create klaxons and a big flashing sign announcing the doppelganger's presence would be the personal choice).
The 4th-level spell polymorph can indirectly reveal that a creature is a shapechanger like a doppelganger, because:
The spell has no effect on a shapechanger
So an attempt to transform a doppelganger using polymorph would automatically fail, even if the doppelganger was a willing target. The doppelganger may well not be aware of this limitation of the magic, so this scenario could conceivably arise by accident - but it could be used deliberately by a suspicious caster who understood this property of the spell.
The 5th-level spell commune would likely allow a suspicious cleric to confirm the identity of other party members:
You contact your deity or a divine proxy and ask up to three questions that can be answered with a yes or no. You must ask your questions before the spell ends. You receive a correct answer for each question.
It only guarantees 3 correct answers a day, as multiple castings can go awry - optimising which questions to ask to most quickly narrow down who is a doppelganger is left as an exercise for the reader. Of course, the spell leaves open the possibility that any particular question might be outside the deity's knowledge, so you don't have to allow this method to work.
Any creature with the special ability of truesight can, amongst other benefits, identify the true form of shapechanged or polymorphed creatures:
A monster with truesight can, out to a specific range, [...] perceive the original form of a shapechanger or a creature that is transformed by magic.
Some kinds of monsters naturally have truesight, but there are also a few ways for players to gain the ability:
- The 6th-level spell true seeing grants truesight to the subject
- The rare magical item gem of seeing can grant truesight a few times a day
- The legendary magical item crystal ball of true seeing grants truesight when it used to cast the scrying spell
- The Epic Boon of Truesight grants permanent truesight (though that is suggested as a reward for 20th level characters)
- The Eye of Vecna artifact grants truesight when attuned
- The Sword of Zariel artifact (from Descent into Avernus) grants truesight when attuned
Some of the monsters that have truesight could be summoned by other spells and requested to identify any shapechangers they detect, the simplest options being:
- A couatl, can be summoned by the 7th-level spell conjure celestial (probably the friendliest option)
- A glabrezu, can be summoned by casting summon greater demon with an 8th-level spell slot (but a very dangerous option as demons are hard to control)
- An orthon, can be summoned by casting infernal calling with a 9th-level spell slot (slightly less risky than the demonic options as devils are a bit more restrained, but still dangerous)
And finally the 9th-level spell shapechange could directly transform the caster into a creature that naturally has truesight.
A 15th level warlock can take the Witch Sight eldritch invocation, which essentially grants a diet version of truesight:
You can see the true form of any shapechanger or creature concealed by illusion or transmutation magic while the creature is within 30 feet of you and within line of sight.
It doesn't provide the other benefits of truesight, but those are irrelevant to finding doppelgangers!
A lantern of tracking (a common magical item from Icewind Dale: Rime of the Frostmaiden) made to detect monstrosities will indicate the presence of a doppelganger:
While the lantern is within 300 feet of any creature of that type, its flame turns bright green.
It doesn't pinpoint the creature though, and the detection radius is broad enough to make it a bit tricky to triangulate the signal just by moving around in most environments, though with a sufficiently large empty space available it could be used to prove whether or not any given individual is secretly a monstrosity in disguise.
When a doppelganger impersonates an individual, they may have the right appearance, but they don't necessarily have the physical, social and mental qualities of the subject they are impersonating.
For social characteristics, they depend on good old-fashioned stalking or their ability to read their victim's mind to gain that information, both options which might not be available at the time. As per the Monster Manual:
A doppelganger’s adopted form allows it to blend into almost any group or community, but its transformation doesn’t impart languages, mannerisms, memory, or personality.
If the doppelganger is impersonating a specific person that the party was previously familiar with, they might notice that something is off about their friend if the doppelganger does a poor job. Mechanically, you could make Charisma (Deception) checks opposed by the party's Wisdom (Insight) - the doppelganger's ability to read minds can grant advantage on that check, but only against one person at a time - if they're interacting with more than one person at a time, they will have a harder job deceiving the group.
The doppelganger's mind-reading ability allows them to detect surface thoughts so it is difficult to rely on any method that involves secret information to identify an impostor - the doppelganger can simply read their interlocutor's surface thoughts and will know what answer is expected when a question is asked of them. However, resourceful individuals who are aware of the doppelganger's mindreading potential could contrive workarounds for this, such as using an ignorant third party as a go-between for asking questions and reporting answers, or ensuring that line of effect to the potential doppelganger is broken via a sufficiently impermeable barrier (such as a lead sheet).
If the doppelganger is impersonating someone who was previously unknown to the party, so they have no baseline to compare against, they'll have no cause for suspicion. However, you could throw a clue to the players by having them meet an NPC who does recognise the person the doppelganger is meant to be, and have the interaction of the NPC and the doppelganger be suspicious - perhaps the doppelganger does not recognise the NPC and has to obviously bluff their way through the interaction, or the doppelganger's imperfect behaviours make the NPC suspicious and wary and the players can pick up on that.
Besides the problem of convincingly impersonating a specific person, however, there's no generic social tell that's common to doppelgangers. They are intelligent creatures who live amongst humanoid societies and would understand the social norms and common knowledge of those societies, so can convincingly pass as a generic member of the populace.
A doppelganger can also be given away if their physical capabilities don't match up with the form they have assumed. As their Shapechanger ability states:
Its statistics, other than its size, are the same in each form.
Doppelgangers are very dextrous, which aids them in adroitly impersonating the gait and mannerisms of a studied victim, but they have only average strength. A doppelganger who has assumed the identity of a particularly strong humanoid may look very muscular, but be unable to back up the appearance of that physique with any displays of genuine strength. Under pressure, such as in combat, they may react with more agility than expected.
Additionally, they retain their darkvision when polymorphed, but don't gain any of the assumed form's special abilities. So, for example, a person who is seemingly human but appears to be able to see in the dark would be suspicious, as would an elf who needs to sleep and cannot trance, or a firbolg who suddenly can't use their natural magical abilities. If impersonating someone who is known to be a spellcaster, their inability to cast any spells would also be a cause for concern.
A smart doppelganger who has the luxury of choice would presumably try to avoid impersonating anyone who has physical or magical abilities that they cannot reproduce, but particular circumstances might force them to assume such a form, in which case the mismatch between their identity and their abilities could be noticed by perceptive characters, and could be used as a test under suspicion.