Thirsting Blade only works with pact weapons
The Thirsting Blade eldritch invocation simply states (emphasis mine):
You can attack with your pact weapon twice, instead of once, whenever you take the Attack action on your turn.
Although similar to Extra Attack, this feature does not work the same way; it clearly states that you attack twice with a pact weapon, which means the feature does not apply to any non-pact weapon. Therefore, making an attack with a non-pact weapon does not qualify for Thirsting Blade, and you would be unable to make any further attacks with that same Attack action.
Contrast this with the UA Artificer's Arcane Armament feature1 (emphasis mine):
Starting at 5th level, you can attack twice, rather than once, whenever you take the Attack action on your turn, but one of the attacks must be made with a magic weapon, the magic of which you use to propel the attack.
This feature notes that only one of the attacks needs to be with a magic weapon, so you would be able to make the other attack with any weapon when using Arcane Armament. As you can see, this is very different wording from that of Thirsting Blade.
1. Suggested by Someone_Evil. This feature does not appear in the final version of the artificer published in Eberron: Rising from the Last War.