This question got me thinking about the mechanics of the Magic Initiate feat when you have class levels in the class into which you were inducted.
Among the classes you can pick with this feat are the Bard, Sorcerer, and Warlock, which know a limited number of spells and all have the same statement under Spells Known.
Additionally, when you gain a level in this class, you can choose one of the [class] spells you know and replace it with another spell from the [class] spell list , which must also be of a level for which you have spell slots.
The term "learning" is used both to describe new spells acquired on level-up and the spell acquired through the feat. This would seem to indicate your Magic Initiate spell counts among the spells you know, and is thus eligible for retraining if you learned it as a spell of your class.
An odd change in wording occurs during the Magic Initiate feat description (emphasis mine):
In addition, choose one 1st-level spell from the same list. You learn that spell and can cast it at its lowest level.
This generalization in describing the learned spell's level seems to imply that there exists a scenario where it would not be appropriate to restrict the spell to being cast as a 1st-level spell. Could one of the charisma casters retrain Magic Initiate into a higher level spell, which they could then cast once per long rest without using a slot?
This would turn an otherwise unexciting feat into, effectively, a Mystic Arcanum for Bards and Sorcerers. This seems too powerful to be intentional, but the feat description appears to have gone out of its way to permit it.