Up to the DM
Thrown is a defined weapon property. This is explained on page 146, PHB under Weapon Properties
Many weapons have special properties related to their use, as shown in the Weapons table.
Thrown is one of the properties:
Thrown. If a weapon has the thrown property, you can throw the weapon to make a ranged attack. If the weapon is a melee weapon, you use the same ability modifier for that attack roll and damage roll that you would use for a melee attack with the weapon.
Not anything you can throw to attack is a thrown weapon, only weapons with the property. For example if you use an improvised weapon it's a thrown weapon that does not have that property:
If a character uses a ranged weapon to make a melee attack, or throws a melee weapon that does not have the thrown property, it also deals 1d4 damage.
Usually, you only use common English definitions to fall back upon for when there is no defined game game term, like there is for thrown weapons. Alas, there is no defintion of what is a defined term either, and very few sections actually call out they talk about the game's defintions (like what counts as an attack, or what counts as an object).
Language for weapon property in game features
The text snippet of Thrown Weapon Fighting that you cite does not say "a weapon with the thrown property", but neither do some other features that clearly refer to a weapon property, for example the Defensive Duelist feat about weapons with the Finesse property is worded like this
When you are wielding a finesse weapon with which you are proficient (...)
Great Weapon Master says:
Before you make a melee attack with a heavy weapon that you are proficient with (...)
If that were not the weapon property, any weapon that weighs enough to be considered heavy would work with GWM (and the heavy property is really not about weight, it is badly named and is about unwieldiness).
However, there are also counterexamples. The Great Weapon Fighting style says:
The weapon must have the two-handed or versatile property for you to gain this benefit.
So the use of language is not consistent. In some cases, the game does say "with the xyz property", in others it does not. Wether the word "property" is present does not really give us conslusive evidence if the weapon needs to have the property or not.
Finally, the full text of Thrown Weapon Fighting (Tasha's Cauldron if Everything, p. 42) is:
You can draw a weapon that has the thrown property as part of the attack you make with the weapon.
In addition, when you hit with a ranged attack using a thrown weapon, you gain a +2 bonus to the damage roll.
Is the "property" language omitted in the second paragraph because it would be awkward to read otherwise, or is it omitted because this benefit purposefully applies to any thrown weapon, not just to those with the property? For feats, if the rules want to separate out independent benefits, they use bulleted lists. This is not done here, but there are two separate paragraphs.
The paragraphs, which serve to represent a separate thought could be used as an argument that the second benefit works for any weapon you throw, not just those with the Thrown property, like in the first paragraph. They otherwise could have worded the features as something like:
When using a weapon that has the thrown property, you gain the following benefits: (...)
Conclusion
I think in the end, this is really too ambiguous a case. There is no clarification in Sage Advice for it. As in other cases where the rules do not provide a clear answer, your DM must decide if they allow this for any thrown weapon (in the common English sense), or just for those with the Thrown property.
