Dungeons & Dragons has a long, long history of devils, and earlier editions gave them a lot more detail. So far as I know, all of the following is 5e-compatible (as in, it won’t contradict anything in 5e), but these details haven’t necessarily all been called out in 5e (yet?).
The term “devil,” by and large, refers to the outsiders found in the Nine Hells of Baator, the lawful-evil outer plane. This contrasts with the chaotic-evil demons in the Abyss and the neutral-evil yugoloths in Hades. There are also evil outsiders not from the outer planes at all—at least in prior editions, rakshasas were evil native to the material plane rather than any of the lower planes.
However, there are more than one “race” of devil. The most common in present-day Baator are known as the baatezu, but some devils like chain devils (kytons) are not baatezu but separate. For the record, hell hounds are another example of this: they are devils, but are not baatezu. On top of those distinctions, long, long ago, there were the ancient baatorians; see our Q&A on the ancient baatorians as of the 3.5 revision of D&D.
Ultimately, what makes a creature a devil is being formed from the substance of Baator. Unsurprisingly, this usually happens in the Hells, but doesn’t quite have to, and again, most outsiders that form in the Hells are devils, but this doesn’t quite have to be the case. For example, several gods have their realms within the Nine Hells (or, at least, did; I believe 5e may have changed this), and were capable of creating non-devil servants for themselves.
What makes a creature a baatezu in particular is that they are part of the baatezu development tree. All baatezu start out as lemures, and turn into the various advanced baatezu from there. And baatezu has been a defined, mechanical status in some editions of D&D. For example, in the 3.5e revision, the glossary entry on the baatezu subtype offers the following properties common to all baatezu:
- Immunity to fire and poison.
- Resistance to acid 10 and cold 10.
- See in Darkness (Su): All baatezu can see perfectly in darkness of any kind, even that created by a deeper darkness spell.
- Summon (Sp): Baatezu share the ability to summon others of their kind (the success chance and type of baatezu summoned are noted in each monster description).
The most prominent of these, to my mind, are being Lawful Evil, and the strong resistances to fire and poison and lesser resistances to acid and cold and the ability to see in even magical darkness. In fact, the ability to see in magical darkness has sometimes been made available to non-devils through spells called devil sight, which kind of emphasizes how iconic that ability is. Not mentioned here is a vulnerability to silver and/or resistance to non-silver weapons, which is also a big part of the devils in previous editions. The second edition even had a bit about “stinger” being slang for a silver piece on the planes, because holding one was unpleasant for devils and trying to trade with one in silver was a good way to make an unpleasant being angry with you specifically.