Double those dice on a critical!
The Vorpal Sword states:
A creature is immune to this effect[immediate beheading] if it is immune to slashing damage, doesn't have or need a head, has legendary actions, or the GM decides that the creature is too big for its head to be cut off with this weapon. Such a creature instead takes an extra 6d8 slashing damage from the hit.
The order of events is:
- Player rolls a natural 20
- Vorpal Sword property activates
- If beheading is not possible, sword delivers 6d8 damage
- A 20 is also a critical, which would then activate the double dice
The extra 6d8 is the damage dealt upon rolling the Natural 20 if , and damage dice are doubled on a critical hit.
Crawford agrees via Twitter:
Querent: Vorpal Sword: Alternative 6d8 damage if you cannot decapitate. Does the 6d8 get doubled?
Jeremy Crawford: Roll those dice twice!
Another view
It is a bit weird that the activation for damage is the same as that for Critical Hits. Why not just have the damage be 12d8 to account for critical hits (and say that it isn't doubled.)
On one side, I think it's reasonable to say that damage is dealt outside of the critical hit - but on the other, non-immune to beheading creatures are killed outright and 6d8 is unlikely to kill something outright at the level at which a character has a this Legendary Magic Item.
12d8 damage actually seems more reasonable to deliver if it's not auto-killing. It's just weird to have the damage occur with the same roll as a critical and then immediately doubling the dice.