Icyfire's answer is perfectly sufficient with regard to 5e and duergar in isolation. This answer will attempt to dive into lore as well as explain the 'exception' of the svirfneblin.
Duergar sensitivity appears to come from their evil nature
To answer the question of why Duergar are sensitive to sunlight, we can go back to the first three Underdark races described in First Edition (drow, kuo-toa, and svirfneblin), some of which are averse to light and some which are not. Adding in the duergar and analyzing similarities and differences, we can attempt to determine what the 'cause' of the duergar aversion is.
Although presaged in G1, the drow make their first "on-screen" appearance in G3, where we learn that
Ages past, when the elvenfolk were but new to the face of the earth, their number was torn by discord, and those of better disposition drove from them those of the elves who were selfish and cruel. However, constant warfare between the two divisions of elvenkind continued, with the goodly ones ever victorious, until those of dark nature were forced to withdraw from the lands under the skies and seek safety in the realm of the underworld. Here, in lightless caverns and endless warrens of twisting passages and caves hung with icicles of stone, the Dark Elvenfolk, the Drow, found both refuge and comfort. Over the centuries they grew strong once again and schooled themselves in arcane arts. And if they were strong enough to face and defeat their former brethren in battle, the Drow no longer desired to walk upon the green lands under the sun and stars. They no longer desired a life in the upper world, being content with the gloomy fairyland beneath the earth that they had made their own. Yet they neither forgive nor forget, and above all else they bear enmity for all of their distant kin - elves and faeries - who drove them down and now dwell in the meadows and dells of the bright world. Though they are seldom if ever seen by any human or demi-human, the Drow still persist, and occasionally they enter lower dungeon levels and consort
with other creatures in order to work out their schemes and inflict revenge upon those who inhabit the world above.
This passage (G1-3 p.29) sets the tone for everything that comes later. The 'darkness of the drow' is their moral darkness; their aversion to light is physical, but is also a tangible reminder of their evil. This idea probably goes back even beyond our Manichean cultural heritage. The TV Tropes page Weakened By the Light is a good place to start to understand the themes that Gygax was drawing on here, and the sub-page providing details on TableTop Games has a long section on D&D that is a good first survey.
Drow have their alignment listed as Chaotic evil and "have superior infravision of the [120 foot] range variety" but are both averse to light and receive substantial mechanical penalties while in it.
(G1-3 p.30, emphases in the original):
Because the Drow have dwelled so long in the dark labyrinthine places under the surface of the earth, they dislike bright light. They will not venture forth into daylight except on the cloudiest, gloomiest days. If within the radius of a light or continual light spell the Dark Elves are 90% likely to be seen, they lose 2 from their dexterity, and all attacks made are at -2 on "to hit" dice rolls - +2 on saving throws against such attacks as applicable. If they are attacking targets inside the radius of light or continual light spells, the bright illumination causes them to suffer a "to hit" dice penalty of -1 , and the converse +1 on saving throws against such attacks is awarded to the target creatures. If bright light exists, it is 75% likely that the Drow will retire from the situation because of the illumination, unless such retreat imperils one of their number or would otherwise be harmful to their desired ends or expose some important feature to the light-bringing intruders. In any event, such light sources as torches, lanterns, magic weapons, fire beetle essence, or faerie fire do not adversely affect the Dark Elves' performance.
As the GDQ series progressed, we were introduced in D2 to both Kuo-Toa and Snirfneblin.
Kuo-Toa, like drow, once lived on the surface but were driven from it.
(D1-2 p.25, with emphases in the original):
The ancient Kuo-Too People once inhabited the shores and islands of the upper world. As the race of mankind and its associate species grew more and more numerous and powerful, the men-fish were slowly driven to remote regions. Continual warfare upon these evil, human-sacrificing creatures threatened to totally exterminate the species, for a number of powerful beings were aiding their sworn enemies, mankind. Some Kuo-Toans sought refuge in sea caverns and secret subterranean waters, and while their fellows above were being slaughtered, these few prospered and developed new characteristics to match their lightless habitats. However, the seas contained other fierce and evil creatures with designs of their own, and the deep dwelling Kuo-Toans were eventually wiped out, leaving only those in the underworld to carry on. These survivors were unknown to men, and mankind eventually forgot the men-fish entirely. Even the word goggler, a term used derisively for their ichthyoid foes, lost its meaning. But the Kuo-Toa People remaining in their underworld places did not allow memory of the past to lapse, and woe to the hapless human who falls into the slimy clutches of the Kuo-Toans! Now the Kuo-Toa People are haters of sunlight and are almost never encountered on the surface of the earth.
Like the drow, the kuo-toa have penalties in light for the naturalistic reason that they now dwell underground, but they hate it because they are evil, and evil hates the light. Their alignment is listed as Neutral evil (chaotic tendencies).
Further (D1-2 p.27, emphasis in the original):
Although their eyes are set on the sides of their heads, Kuo-Toans have excellent independent monocular vision, a very wide degree of field (180°). and they are able to detect movement even though a creature is normally invisible due to magic, astrally projected or ethereal. Thus, only complete
motionlessness can avoid sight detection by a Kuo-toan. They see into the infra-red and ultra-violet spectrums. Also, vibrations within 10' can be felt by these creatures, so they are surprised only on a 1 (d6)...all forms of illusion are useless against them. However, Kuo-Toons hate bright light-such as from a light spell - and fight at -1 on "to hit" dice rolls when
exposed to such illumination...
In contrast to the drow and kuo-toa having been driven from the surface into the dark, it is not clear where the geographic origin of the Svirneblin race is, for the description of them given in D2 presents them as relatives of surface gnomes who simply live underground: we are not told whether they started there or moved there (D1-2 p. 28):
Far beneath the surface of the earth dwell the Svirfnebli, the Deep Gnomes, a race related to the gnomes of the bright world. Small parties of these demi-humans roam here and there in the underworld mazes of small passageways, always in search of gem minerals. Their realm is in a region unknown, but thought to consist of a closely connected series of vast caverns in which thousands of these diminutive creatures labor for their king. Only males have ever been seen, and those only in very deep places
beneath the ground.
Svirfneblin "have infravision to 120 [feet]" but nowhere in their description does it mention them being averse to light (D1-2 p. 28). However, it is very notable that unlike the previous two races, they are not evil: their alignment is listed as "Neutral (good tendencies)"
The Duergar did not originate in the GDQ series, but instead were first introduced in Monster Manual II. Like the Svirfneblin, they are not specifically listed as being refugees from the surface, and for all we know may have always lived underground (1eMM2 p.61):
The duergar are the seldom-encountered race of evil dwarves. They are also known as gray dwarves or gray ones. Malicious in the extreme, duergar dwell in the subterranean depths where their evil need not withstand the light. Normally but a handful of these wicked creatures will be discovered, but occasionally a lair will be discovered with hundreds of duergar there.
Interestingly, while "Duergar have infravision of [the 120 foot] range," they aren't originally listed as having any mechanical penalties to performing in conditions of light. Rather, we are told only that they dwell "where their evil need not withstand the light" and have their alignment listed as Lawful evil (with neutral tendencies).
Thus, to answer the question of why Duergar are sensitive to sunlight, we now have a good sample set of four Underdark races (drow, kuo-toa, svirfneblin, and duergar), three of which are averse to light and one which is not. By comparing them, we can attempt to determine what the 'cause' of the duergar aversion is.
All four of the races have infravision to 120 feet, so simply having 'better than normal infravision' is not sufficient to explain this as a cause of light aversion in the lore (even though that this is the explicit cause for them in 5e).
In an earlier answer to this question, I had thought that it was the snirfneblin possession of ultravision that protected them from sunlight sensitivity. However, upon discovering that the kuo-toa also have ultravision and yet still have penalties in the light, I was forced to discard this idea.
Only two of the races (drow and kuo-toa) are explicitly refugees from the surface lands, while duergar and svirfneblin are not. Thus, this cannot be the cause of the duergar aversion to light.
Finally, we note that the three races listed as hating sunlight are also the ones that are explicitly evil, while the svirfneblin are not. This, then, is the most parsimonious lore explanation of why duergar don't like light: they are evil, and evil hates the light.
Fifth Edition
In converting Drow, Kuo-Toa, Svirfneblin, and Duergar to 5th edition, the game designers have stayed true to the first edition roots of the races, in that they all have superior Darkvision, but only the 'historically evil' races have been given Sunlight Sensitivity (a caveat here is that the 5e designers likely also considered how the races were statted out in 2e, 3e, 3.5e, and 4e, which I am not considering here. In particular, I don't know at what point the duergar went from being averse to light to actually having a mechanical penalty while in it).
As to whether this sensitivity is physical or moral in 5e, consider this question-and-response from Jeremy Crawford:
Q:Underdark Sunlight sensitivity. Does bright light from torches,
light cantrip etc cause the disadvantage or only sunlight?
A: The text of Sunlight Sensitivity specifies that sunlight is the
problem for the creature. Bright light in general is fine.
That is, it is not the brightness of the light per se, but the fact that it is from the sun that is debilitating. Even as 5e WoTC moves away from the concept of 'evil races', the Sun as an ancient symbol of Good still clings to the system.
