I want to make a fighter whose primary weapons are his unarmed attacks using the Tavern Brawler feat. If I take the Two Weapon Fighting style would that work with 2 unarmed strikes? In the weapons table it doesn't have the light property generally required for fighting with two weapons, but they're fists.
The rule for Two Weapon Fighting is as follows:
When you take the Attack action and attack with a light melee weapon that you’re holding in one hand, you can use a bonus action to attack with a different light melee weapon that you’re holding in the other hand.
The errata for the PHB states that unarmed strikes are not weapons, so you can't use Two Weapon Fighting at all.
The rule on unarmed strikes should read as follows: “Instead of using a weapon to make a melee weapon attack, you can use an unarmed strike: a punch, kick, head-butt, or similar forceful blow (none of which count as weapons). On a hit, an unarmed strike deals bludgeoning damage equal to 1 + your Strength modifier. You are proficient with your unarmed strikes.”
You need two weapons to perform Two-weapon fighting
The rule is fairly clear on this. From the Basic Rules:
When you take the Attack action and attack with a light melee weapon that you're holding in one hand, you can use a bonus action to attack with a different light melee weapon that you're holding in the other hand.
You need to hold a light melee weapon in each hand in order to be eligible. Unarmed strikes aren't weapons nor light. Although realism is not the best argument for game design, I want to respond to this concern of yours,
In the weapons table it doesn't have the light property generally required for fighting with two weapons, but they're fists.
that swinging a sharp knife or a dagger in a manner to be damaging requires less, not more force than swinging a fist. The force equivalent to a light slap gets deadly when it's concentrated on a sharp enough edge.
Will it break anything if I houserule it?
Not in the balance sense, but keep reading! Unarmed strikes deal paltry damage, and are likely to remain unused anyway apart from gimmick builds. It might grant characters additional tactical mobility if they use their other hand periodically for thrown weapons or other such uses that only require the hand for moments at a time. It will also grant some extra utility to most melee-oriented characters if you rule that a character needs no free hand to perform Two-weapon fighting unarmed, eg with a kick.
However, the paltry damage output --- just one damage point assuming no feats --- makes rolling for unarmed strikes an inefficient use of table time. Extra rolling for minor effects like this slows down encounters as more time is lost to rolling effects that don't meaningfully affect combat outcomes. That is the main reason I recommend against houseruling this.
No you cannot use two weapon fighting to add your ability score modifier to your off hand empty strike. Only a monk can do that. You'd need to start off a couple monk levels or just stay monk. Just like an oath breaker Palladin, you could roleplay yourself as a tavern brawling, pit fighting monk. You don't have to picture yourself as some bald dude in a robe with.
To answer your question: Yes, you can two weapon fight unarmed, it just requires you to be a monk and utilize their class mechanics to game the system appropriately.
I think what you're looking for is a brawling monk rather than a fighter. As per PHB pg. 78, emphasis mine:
You can roll a d4 in place of the normal damage of your unarmed strike or monk weapon. This die changes as you gain monk levels, as shown in the Martial Arts column of the Monk table.
What this means is that the two weapon fighting style directly benefits your monk, as does the duelist style. All you have to do is wield the appropriate weapons in your hand. In addition, thanks to the flavour below this section about specialized weapons, you can simply call the daggers you are holding a special kind of light, finesse bladed knuckle weapons.
So, when making this fighter in order to maximize their unarmed damage, I would recommend taking the 1st level as fighter for the two weapon fighting feature, and the rest of your levels go monk to maximize the unarmed damage to supplement the weapon damage (as well as gaining Flurry of Blows for the extra attacks.) While the class name might be monk, you can definitely roleplay this as a straight tavern brawler.
I think this just makes sense to allow. If you're proficient with your unarmed strikes, anyone that knows how to throw a punch knows how to follow up with another punch with his other hand. I mean granted any character could just always two weapon fight with an unarmed strike as their off-hand attack, but what real advantage is 1 extra point of damage that you have to roll for anyway on top of not getting a damage bonus, anyway?
As a DM, I would say an unarmed strike is light, and any person with two arms that has even a rudimentary knowledge that punch = damage won't win any fights by only using one arm to punch. It's 1 point of damage, whoop-dee doo. Or — oh no — tavern brawler with that deadly 1d4 (that you could also do with a dagger, so what's the real difference other than damage type? Nothing.)
So what if the wizard wants to suddenly start boxing? He's not going to do anything for damage. And as for "just play a monk!" argument, maybe some people like wearing armor and not being wisdom dependent. I see no game-breaking aspect of allowing a fighter to use two weapon fighting with an unarmed strike and his damage die doesn't even go up. A plate armor juggernaut wouldn't be any more ridiculous in strength than a standard monk. And if a player being unable be disarmed(even though a monk has that same possible advantage) is really that big of a game breaker then you need more oozes.