So could I cast Vampiric Touch (this last 1 minute) and using my unarmed attack bonus do Vampiric Touch damage + unarmed strike damage?
Nope. Vampiric Touch is a Cast a Spell Action, making an Unarmed Strike is an Attack Action. Attacks with Vampiric Touch use your Spellcasting Attack Bonus, as it is a spell. The subsequent attacks with Vampiric touch are likewise specific actions that precludes you from making an Attack Action with your unarmed strike.
If I can't would it be a reasonable house rule?
Also no, an extra 3d6 necrotic damage along with your unarmed strikes is a huge imbalance, that's like an instant Sneak Attack as a 5th-level rogue with no need for ally adjacency or Advantage, not to mention you can up-cast the spell to keep up with the rogue. It's additional damage (10.5 average) is also better than a Paladin's Divine Smite (9 average) per hit, and it consumes only 1 spell slot that you can then use up to 10 times. Oh, and it heals yourself on top of all of that.
Sacrificing your +2 attack bonus for +3 increase in damage might sound reasonable to you now, but there are too many ways to improve those bonuses including picking up certain feats, class features, and magic items. Soon, your houserule will become overpowered.