Technically Yes, but practically No.
Whirlwind Attack uses the phrase "make a melee attack". Note that this is the same wording as Opportunity Attack's "make one melee attack", and even the Attack action's "make one melee or ranged attack". It stands to reason that any melee attack you can make using the Attack action, you can also make using Opportunity Attacks or with the Whirlwind Attack action.
But what attacks can you make using the Attack action?
Technically, the rules are surprisingly silent on this.
We assume that we can use a weapon we are holding, but this is never actually stated in the rules.
Grappling and Shoving are specifically called out as being possible using the Attack action. To me, this implies you can also make them using Whirlwind Attack, but it's not exactly clear if this is an exception or a clarification.
Let me ask you this: If a monster has an attack listed in its stat block under "Actions", can it use that attack as an Opportunity Attack? Can a Wolf use its claws against a fleeing opponent, potentially knocking them prone? I'm going to hazard a guess that most DMs rule that they can, without even considering the question.
Does it matter if the attack listed is a melee weapon attack or a melee spell attack? Can a Specter use its Life Drain attack as an opportunity attack? If a Specter was targeted by the Haste spell, could it use the additional Attack action to use Life Drain? If so, then a Specter that somehow uses Whirlwind Attack could indeed use Life Drain against multiple opponents.
According to the rules, there is no difference between a melee weapon attack and a melee spell attack for the purposes of the Attack action, nor Opportunity Attacks, nor Whirlwind Attack. If you can use one, you can use them all.
Therefore, a Hunter who has an unrestricted melee spell attack option (perhaps they have been Shapechanged into a Specter) can indeed use it with Whirlwind Attack, just as surely as they can use a held weapon.
However
That said, just because you can make an attack, doesn't mean you can perform an additional action that allows you to make that attack. Technically, ALL attacks require you to perform an action (or similar) to actually make the attack, so this isn't terribly helpful. However, I suspect that most DMs would not allow you to make the Cast a Spell action in the middle of the Attack action, therefore you would also not likely be able to cast Inflict Wounds as part of a Whirlwind Attack.
Vampiric Touch, on the other hand, is somewhat more reasonable to allow, as the spell effect grants you an attack that you can use on subsequent turns (the spell description does state that this attack requires an action, but like I said, ALL attacks require some kind of action, so if this is disallowed then as written you can't use a weapon either).
Hypothetically, if you use the Ready action to cast Inflict Wounds and hold the effect until the trigger occurs, you might then be considered to have the attack available, and could then Action Surge to use Whirlwind Attack with the readied attack from Inflict Wounds, but this would require a very specific interpretation of the rules that I suspect would not normally be allowed.