The number of spells in your spellbook does little to affect balance
The main reason for this is that one of the major limitations as a wizard is how many different spells you can prepare in a day. After you prepare generally useful spells like Armor, Shield, Invisibility, Misty Step, Counterspell or Fireball, you typically have preciously few "free" preparation slots left for spells with more narrow applications.
It does not matter if you have a spellbook with dozens of other spells then. I speak from practical experience: I have copied many spells to my book that in the course of eight levels worth of play I have never even once used; and that even though I make it a point to prepare such subpar spells on purpose, to see if I can find some fun, creative application.
Maybe the most useful spells to add are rituals, which tend to see a lot of use as they do not consume spell slots if cast as a ritual. However, there are relatively few rituals to be had overall, and their effect tends to be utility, so they are certainly not unbalancing in combat.
Flexibility
The added flexibility from extra spells can be worthwhile, because you can balance your memorized spells to match the adventure you are playing, if you know what to expect. In a battle raid against a goblin tribe, memorize sleep, shatter, or freezing sphere. In an city detective adventure, memorize suggestion, detect thoughts and locate object. On a day securing your tower, memorize magic mouth, arcane lock and major illusion.
If you are only interested in the effect on combat encounters, then the benefits of this flexibility matter even less. If you already stuff all your memorization slots with combat spells, having even more combat spells in your book will matter relatively little.
Whats more, this added flexibility is fully accounted for in the description of the wizard class, which says:
On your Adventures, you might find other Spells that you can add to your Spellbook.
There is no limit imposed on this way of adding spells, neither in the PHB nor in guidance in the DMG. That is, the game accepts that you could add additional spells to the spellbook without limit, and without materially affecting game balance. Having a lot more spells in the book will not make a wizard "far more powerful than other PCs", it's part and parcel of the wizard class. (And from my experience, it also is very costly; my wizard is constantly broke, a further downside of adding extra spells).