Yes: damage type fundamentally attaches to instances of damage, not anything else
Damage Resistance and Vulnerability (PHB p. 197) says:
Multiple instances of resistance or vulnerability that affect the same damage type count as only one instance.
Taken in isolation, the natural interpretation of this sentence is that the damage type is attached to the resistance, and this sentence prevents "resistance to fire damage" from one source from stacking with "resistance to fire damage" from another source because they both name the same type of damage. On the other hand, "resistance to fire damage" is not the same type as "resistance to nonmagical damage", so this sentence would not apply to that pairing. Likewise, in the present case invulnerability's "immunity to all damage" is not the same type as "immunity to fire damage", so Fiery Soul would not apply.
There's just one problem: this natural interpretation is abrogated by the example that immediately follows in the text.
For example, if a creature has resistance to fire damage as well as resistance to all nonmagical damage, the damage of a nonmagical fire is reduced by half against the creature, not reduced by three-quarters.
In the given example, despite one resistance being to "fire damage" and the other resistance being to "all nonmagical damage"---neither being the same as, or even a subset of, the other---they are considered to be affecting "the same damage type", which is used as justification for the resistances not stacking. The implication is therefore that damage type fundamentally attaches to actual instances of damage. Everything else, such as resistances, is processed through this attachment.
If we apply this to the present case:
invulnerability states you are immune to "all damage", not "fire damage" or "all damage types"
the fact that invulnerability does not literally specify "fire damage" does not matter---since the actual damage dealt by the Pyromancer's spell falls under both invulnerability's "all damage" and Fiery Soul's "fire damage", they are considered to be "the same damage type" in this instance and therefore Fiery Soul applies. An awkward reading, yes---but awkward in the same way that the paragraph from Damage Resistance and Vulnerability is awkward.