Do the extra attacks from the Fencing Master feat and the extra attack 5th level class benefit stack? Do I get to attack 3 times in one turn when having both of these?
Yes, you get can get 3 attacks, but with a -5 penalty to each. Rarely worth it.
At 5. level as a paladin you can make melee 2 attacks as an action. With this feat (wielding the right weapon), you can decide to make one additional attack, but in this case you take a -5 penalty to all three attacks. Doing so is rarely good for you, usually only worths it if you have advantage.*
The main attraction of Fencing Master from an optimization standpoint is the ability to parry the attack with a reaction.
Connecting to your other question regarding an empty hand, even with this feat you benefit from a shield in your other hand.
*The mathematics of when is it better to attack 3 times with -5:
Where H is your hit chance, and D is your damage. The 0.25 is the -5 converted from roll penalty to hit chance. As you can see, if you have a 0.75 chance (in other words you hit on a 6) it makes sense to make 3 attacks.