If a DM allows a player to retrain the feat at all then the character would lose all those extra spells they gained from the feat. If they don't have records indicating what spells they took at each level, the DM would be completely justified in deciding on some random method of choosing which spells are lost. I would go with choosing random spells from the highest level able to be cast each class level being adjusted.
While the section on Feat Retraining doesn't specifically address loss of spells known, the section on Substitution Level Retraining does mention it:
Spellcasting: If the new level changes some aspect of your character's spellcasting ability (for example, if she gains or loses
spells known or spells per day), apply these changes as appropriate.
If she must lose one or more spells known, you can select the spells
lost from all legal options.
Just as the Class Features subsection of that same section says to treat replaced class features as if a class feature were retrained under the retraining rules, it can be extrapolated that the above subsection on spellcasting is the appropriate way to handle any loss of spells known due to retraining.
That being said, while the spells cease to be Spells Known, nothing whatsoever says they fade from the wizard in question's spellbook. This is actually alluded to in the Class Feature Retraining section:
Example: Upon gaining a new level, a necromancer could change her school specialization to evocation, thus becoming an evoker. At the
same time, she could also choose to change her prohibited schools from
conjuration and illusion to abjuration and transmutation. Doing so
would cause her to lose access to all spells from the newly designated
prohibited schools. Even if her spellbook contains one or more such
spells, she would lose the ability to prepare and cast them.
Clearly, therefore, the spells remain in the wizard's book, just not as spells known. And in the case of the former Collegiate Wizard, you are not dealing with a school prohibition. Ergo, I can think of no reason, if the DM allowed this retraining in the first place, that the wizard in question could not study those spells like any other spells not yet learned (whether from a friend's spellbook, one captured from an enemy, or found in a treasure hoard) and, with successful Spellcraft rolls learn them the hard way (as opposed to the free, automatic spells gained when leveling).
The DM might make him pay the normal cost for inks and make him recopy the spells, since the old versions in his books were the way a Collegiate Wizard would cast the spell, but unless the DM has grandfathered old 2nd edition spellbook inks, ink is cheap compared to the normal spell acquisition costs of scrolls or access to another wizard's spellbooks. The wizard is already taking two weeks off retraining the feat, what's another week or two relearning the spells that eluded his comprehension when he allowed his classes to fade from his mind. (He really should have paid more attention in the academy, apparently.)
[I personally don't allow retraining of any feat that can only be taken at 1st level, as the whole point of those is they represent something you learned because of your background, during your youth, something you were born with, etc., something which I don't feel you can just retrain. Collegiate Wizard is, in my opinion, an especially good example of a feat that should not be allowed to be retrained.
Mages who have trained in a dedicated college or academy learn
different approaches to spellcasting (and gain different advantages
once their training is completed) from characters who apprenticed at
the side of an experienced arcanist.
...
Collegiate Wizard [General]
You have undergone extensive training in a formal school for wizards.
This feat represents the character's entire approach to casting, gained over years of study. Two weeks of retraining (which is all the time feat retraining takes) isn't going to change that, it's an inherent part of their wizard class. I would say that rather than retraining, this would require a rebuilding quest to redo 1st level for the character without the feat. (And possibly with a different 1st level only feat, depending on the quest.)]
