Criticals double dice that are part of the attack
Short answer: if you want the damage to double, word it as "when" you hit with an attack, if you want it to not double, word it as "after" you hit with an attack.
Details and background
Page 196 PHB has the rule for critical hits damage resolution:
When you score a critical hit, you get to roll extra dice for the attack’s damage against the target. Roll all of the attack’s damage dice twice and add them together. Then add any relevant modifiers as normal. (…)
For example, if you score a critical hit with a dagger, roll 2d4 for the damage, rather than 1d4, and then add your relevant ability modifier. If the attack involves other damage dice, such as from the rogue’s Sneak Attack feature, you roll those dice twice as well.
Sneak Attack is explicitly given as an example that doubles, so let's look at how that is worded. That ability says (p 96 PHB):
Once per turn, you can deal an extra 1d6 damage to one creature you hit with an attack if you have advantage on the attack roll.
The deciding factor is if the extra damage dice are part of the attack, or not.
Wording to deal double damage dice
The wording you have in the question is essentially the same as in Sneak Attack, so your ability, like Sneak Attack, would deal double damage on a critical hit.
Other common effects that are doubled are damage from divine smites, damage from superiority dice, damage from hunter’s mark or damage from hex. They use wordings like
when you hit a creature with a melee weapon Attack (Divinde Smite)
If you hit, you add the superiority die to the attack's damage roll (Superiority Dice, Lunging Attack)
you deal an extra 1d6 damage to the target whenever you hit it with a weapon attack (Hunters Mark)
So, if you want your feature to deal double damage dice on a critical hit, you word to say the damage is part of the attack, for example:
when you hit the target with a [include qualifiers here] attack [include other conditions here], you [include "can" if you want it optional] deal an extra XdY [include optional damage type here] damage
Wording to avoid double damage dice
Other extra damage dice, like those from other secondary effects that happen when you damage a creature (i.e. not as part of the attack) are not doubled.
To achieve this, you need to word your feature in a way that makes it clear the damage dice are is not part of the attack. There are of course many, many ways to do this, depending on the mechanics you want to use.
One example is the Psy Warriors Psionic Strike, which states
after you hit a target (...) with an attack and deal damage to it with a weapon, you can expend one Psionic Energy die, rolling it and dealing force damage
Taking this as a template, a straightforward wording would be
after you hit the target with a [include qualifiers here] attack [include other conditions here], you [include "can" if you want it optional] deal an extra XdY [include optional damage type here] damage
The key difference really is using the word after instead of when, to make it clear that the damage happens after the attack, not as part of the attack.