Your familiar is not you.
Let’s start by framing your question as simply as possible:
What is the difference between a PC casting a spell with an attack roll and a range of touch and a PC casting the same spell through a familiar?
“Delivering a spell” does not exist outside of find familiar, and thus is defined here:
When a familiar “delivers” a spell, the game treats what happens as the familiar having casted the spell.
Therefore, in both cases outlined in the question, the PC is the creature that actually casts the spell and expends the appropriate spell slot. However, in the former case, the game interprets this event as such, whereas in the latter case, the game replaces the interpretation that the PC casted the spell with the interpretation that the familiar casted the spell.
The text of find familiar introduces an exception to this general rule which is pertinent to our question:
If the spell requires an attack roll, the modifier applied is that of the PC, not the familiar.
As far as I can tell, this is the only exception.
This argument can be extended to effects like Hexblade’s Curse, which is fortunate because I think we can all agree that Hexblade is OP enough already.
Two more illustrative examples: cure wounds and ring of spell storing.
Cure wounds heals one creature you touch for 1d8 + your spellcasting ability modifier. A familiar does not have a spellcasting ability. Therefore, the game turns to the rules for casting without a spellcasting ability (DMG p. 141):
"If you don't have a spellcasting ability--perhaps you're a rogue with the Use Magic Device feature--your spellcasting ability modifier is +0 for the item, and your proficiency bonus does apply."
Thus, the target is healed for 1d8 + 0.
Although I believe that speculating on RAI is often dangerous and seldom productive, I feel compelled at this point to say: I seriously doubt that this is merely an oversight by the designers, as there aren't very many touch spells in the game, and cure wounds is by far the most frequently implemented.
Admittedly, the rule I quoted from the DMG was clearly written with magic items in mind, but there is no fundamental difference between a spell being cast through a familiar and a familiar casting a spell with the aid of a magic item. To understand this, consider a ring of spell storing (DMG p. 192):
This ring stores spells cast into it, holding them until the attuned wearer uses them...
Any creature can cast a spell of 1st through 5th level into the ring by touching the ring when the spell is cast. The spell has no effect, other than to be stored in the ring...
While wearing this ring, you can cast any spell stored in it. The spell uses the slot level, spell save DC, spell attack bonus, and spellcasting ability of the original caster, but is otherwise treated as if you cast the spell. (emphasis mine)
This effect is very similar to a familiar "delivering" a spell and much less ambiguous. Here, the things that are carried over from the original caster are fully specified, and everything else defaults to the creature using the ring (although I believe that find familiar implicitly preserves the slot level; the inclusion here is due to the fact that the default for spells cast by means of a magic item is to adopt the lowest possible level for the casting, DMG p. 141).
If you are a warlock attuned to a ring of spell storing and you cast, say, a guiding bolt which your party's cleric has already stored in the ring, hitting the creature cursed by your hex spell yields your extra necrotic damage. Conversely, if the cleric is of the life domain and has stored a 1st level cure wounds in the ring, you don't add extra healing from the cleric's Disciple of Life feature when you (the warlock) cast this cure wounds. You do, however, add the cleric's Wisdom modifier (not your Charisma or anything else), as specified in the description of the ring of spell storing. Then again, if the cleric has a familiar (Magic Initiate, multiclassing, etc.) and casts cure wounds through that familiar, he does not benefit from his Disciple of Life feature or his Wisdom modifier, as specified in find familiar.
Finally, consulting the "Proficiency Bonus by Challenge Rating" table (MM p. 8), we see that an associated saving throw DC is 8 + 0 + 2 = 10 (I don't know if such a spell exists, but you could imagine a melee spell attack that forces a save on a hit).