Ability Score Improvements are almost always better for DPR
On 4th level you could raise your attack stat from 16 to 18, or take Dual Wielder. I will assume you have the Two-Weapon Fighting style, while fighting against AC 10, 14 and 18. Criticals are also included in the final DPR.
Damage 2 x (1d6+3) = 13
AC10: Hit chance 80%, 10.75 DPR
AC14: Hit chance 60%, 8.15 DPR
AC18: Hit chance 40%, 5.55 DPR
Damage 2 x (1d6+4) = 15
AC10: Hit chance 85%, 13.10 DPR
AC14: Hit chance 65%, 10.10 DPR
AC18: Hit chance 45%, 7.10 DPR
Damage 2 x (1d8+3) = 15
AC10: Hit chance 80%, 12.45 DPR
AC14: Hit chance 60%, 9.45 DPR
AC18: Hit chance 40%, 6.45 DPR
You can look at it this way, the damage is the same, as on average 1d6+4 = 1d8+3, but after the ASI you have higher hit chance.
Dual Wielder gives you +1 AC too, but if you are Dex primary, an ASI gives you the same, and +1 save, +1 skills, +1 initiative.
If you are Str primary the ASI is much less helpful in this regard.
(But in this case you could just take Great Weapon Fighting, for significantly higher DPR)
Dual Wielder lets you draw two weapons without sacrificing an action. This can be important in cities, but in the wilderness and dungeons you can just run around with one of your weapons already drawn.
Take Dual Wielder if the only magic weapon you have is unusable without Dual Wielder (because it's not light).
+1 to attack and damage from a magic weapon plus the increased die size is better in itself, but if you consider all the monsters resistant or immune to mundane weapons, it is the highest priority.
You should only pick Dual Wielder if:
- the only magic weapon you have is unusable without Dual Wielder
- you can't keep at least one of your weapons in your hand when you expect a fight
- you can not pick an ASI, because
- you are a Variant Human on 1st level
- you have reached 20 in your attack stat already