Warlocks have a relatively limited number of spells known, and (in my experience) the fewest uses of spells throughout a given day in a campaign. I feel with these limits, it would not be unfair to instead of expanding their spell list with the 10 or so spells from each subclass, actually add the spells to the number of spells known. There is precedent for this, as the Sorcerer subclasses from Tasha's Cauldron of Everything both add spells to the number of spells known, and all of the cleric subclasses add permanently prepared spells to their lists. The only exception I think is needed from this house rule is the Genie subclass's access to Wish, which I would consider an addition to the spell list, instead of having 2 spells available for the ninth level Mystic Arcanum.
Because of their limited use of spell slots, expanding the Warlock options of what to do with those spell slots I feel would not hinder the balance of the game. Is there some balance level that I am missing?