All weapon damage rolls add their corresponding modifier to the damage unless a feature tells you otherwise. This is from the rules on Damage Rolls in the Combat chapter of the Basic Rules:
When attacking with a weapon, you add your ability modifier--the same modifier used for the attack roll--to the damage.
Since you're making an attack with the weapon, you add the modifier. It doesn't matter that you're making that attack because of a feat, or with the opposite end of a weapon, or using a bonus action, or so on. Nothing in the feat suggests an exception to the general rule, so add that modifier.
Note that there are situations where the rules allow you to make an attack with a weapon but deny you a modifier to the damage, such as when using two-weapon fighting ("don't add your ability modifier to the damage of the bonus attack, unless that modifier is negative"), but those are exceptions to the general rule.