Update October 2020: No
As user @V2Blast mentions below, the Sage Advice Compendium page 13 explicitly rules "No" for this question. Page 13:
Can I make an attack with one weapon, then draw a second weapon with my other hand and qualify to use two-weapon fighting?
To use the two-weapon fighting bonus action (PH, 195), you must have both weapons in hand when you make the first attack. If you’re instead fighting with two or more weapons as part of the Extra Attack feature, the rule for the two-weapon fighting bonus action doesn’t apply. The rule for that bonus action applies only to itself, not to any other use of two or more weapons in the game
Original Post
No, you cannot draw your dagger and attack with it while retaining your Duelist feature
But you can draw your dagger as part of an attack and then use it.
The core argument behind this answer is that you can only attack as a bonus action with your offhand weapon when you use your action to attack.
When you take the Attack action and attack with a light melee weapon that you’re holding in one hand, you can use a bonus action to attack with a different light melee weapon that you’re holding in the other hand. (Basic Rules, p. 74 / PHB. p.195)
You don't have this bonus action otherwise.
You can take a bonus action only when a special ability, spell, or other feature of the game states that you can do something as a bonus action. You otherwise don’t have a bonus action to take. (Basic Rules p. 69)
You can draw your weapon as part of an attack...
You can also interact with one object or feature of the environment for free, during either your move or your action. For example, you could open a door during your move as you stride toward a foe, or you could draw your weapon as part of the same action you use to attack. (PHB p. 190)
...but that would remove your +2 bonus
When you are wielding a melee weapon in one hand and no other weapons, you gain a +2 bonus to damage rolls with that weapon. (Basic Rules, p. 25)
So, here is what you are trying to do.
You attack
You can use Two-Weapon Fighting to attack with a weapon that is already in your hand. You can subvert this by drawing your weapon as part of your aforementioned attack declaration.
Since you now have another weapon in your hand, you lose the Duelist feature of +2 damage.
If you forgo drawing your dagger, you are not allowed a second attack since the dagger is not in your hand during your attack action. It's either one or the other, you cannot combine two-weapon fighting and duelist in the same attack action. You can dual wield and drop your dagger as a free action before you attack with it to retain your Duelist bonus for the next turn (turn start, attack with no bonus, attack with dagger, turn end, turn start, drop dagger, attack with bonus)
If you need a logical reason for why this has to be, you can think of a duelist as sort of "fencing" with his body turned to the side, putting all his focus on the one weapon. Dual Wielding is less precise and doesn't allow your character to focus on the one attack, but spreads his focus out over two weapons.
One final note, you cannot use your Rapier for Two-weapon fighting because it does not have the light attribute, unless you have the Dual Wielder feat.