The devil would be banished to a harmless demiplane for 1 minute
The Nine Hells is indeed considered to be one plane. It is even possible to travel between the different layers of the Hells without any planar travel/magic (see this Q&A); that said, it is also possible to travel to the Abyss (via the River Styx) from the Nine Hells without any planar travel/magic, and the Abyss is definitely a different plane to the Nine Hells, so by itself that point isn't necessarily proof that the nine layers are all part of one plane.
In the DMG, Chapter 2: Creating a Multiverse, sub heading "Outer Planes", pp. 64-66 describes the Nine Hells. The description heavily implies that it is one plane it given how it is described:
The Nine Hells has nine layers. [...] To reach the deepest layer of the Nine Hells, one must descend through all eight of the layers above it, in order. The most expeditious means of doing so is the River Styx, which plunges ever deeper as it flows from one layer to the next.
However, if we take a look at the Optional Rule: Pervasive Evil (p. 64), it says:
Evil pervades the Nine Hells, and visitors to this plane feel its influence. At the end of each long rest taken on this plane, a visitor that isn't evil must make a DC 10 Wisdom saving throw. On a failed save, the creature's alignment changes to lawful evil. The change becomes permanent if the creature doesn't leave the plane within 1d4 days. Otherwise, the creature's alignment reverts to normal after one day spend on a plane other than the Nine Hells.
The Nine Hells is referred to a plane, singular, not as a collection of planes. That last part is particularly explicit: "a plane other than the Nine Hells." From this, I think we can safely conclude that the Nine Hells is one plane with nine layers, not nine planes or linked demiplanes or anything like that.
For completeness, I will also include further evidence presented by illustro in this answer, which explicitly refers to planes with layers still being considered to be one plane (from DMG, p. 58):
Layers of the Outer Planes
Most of the Outer Planes include a number of distinct environments or realms. These realms are often imagined and depicted as a stack of related parts of the same plane, so travelers refer to them as layers. For example, [...] the Nine Hells has nine layers [...]
So yes, casting banishment on a devil whilst in the Nine Hells would banish the devil to a harmless demiplane, not to somewhere else on the same plane (i.e. a different layer of the Nine Hells), since banishment says (PHB, p. 217):
If the target is native to the plane of existence you're on, you banish the target to a harmless demiplane.