Can spells from the Magic Initiate feat be swapped out for different ones?

Question

For example, a character takes the Magic Initiate feat at level 4, picking spells from the Wizard spell list, because they have been researching and learning all they can about magic (such as a Sorcerer studying their inborn magic or a High Elf expanding on their Cantrip feature). It stands to reason that they would continue their research of magic over their next 16 levels, which indicates that they should learn new spells.

At any point, can the spells chosen through the Magic Initiate feat be swapped out for other spells, according to the same Magic Initiate spell selection rules of two cantrips and one 1st-level spell?

Answer 1

You can't automatically replace the Magic Initiate spells later on

If a rule allows for retraining it would say so. Magic Initiate does not have any such rule.

Continued study would be simulated by Multiclassing

Whatever character your playing is progressing as a main class, and most of their attention is dedicated to that progression. The Magic Initiate feat captures a small amount of attention devoted to another class, but continued study (such that you would acquire new spells) would be emblematic of a multiclass wizard.

Spellcasting

(this section is pending revision upon further debate here)

If you gain a level in a class that has Known Spells for which you also have your Magic Initiate spells, you could use the Spellcasting feature to replace it:

Additionally, when you gain a level in this class, you can choose one of the [class] spells you know and replace it with another spell from the [class] spell list, which also must be of a level for which you have spell slots.

These spells still count as [class] spells since they appear on that class's spell list. As such, it is eligible for replacement by the Spellcasting feature, but will still be subject to the rules that:

Once you cast it, you must finish a long rest before you can cast it again using this feat.

Answer 2

If a Sorcerer took the Magic Initiate feat and used it for Sorcerer spells, thus learning only Sorcerer cantrips and a 1st level Sorcerer spell, the rule for swapping out spells would appear to apply.

Additionally, when you gain a level in this class, you can choose one of the sorcerer spells you know and replace it with another spell from the sorcerer spell list, which also must be of a level for which you have spell slots.

Thus the 1st level spell could be swapped out, but only for another 1st level spell since the feat itself restricts the available level even though the general swapping rule does not (specific rule always wins over general rule).