In D&D 5th Edition the Sorcerer Metamagic ability Empowered Spell states the following:

"When you roll damage for a spell, you can spend 1 sorcery point to reroll a number of the damage dice up to your Charisma modifier (minimum of one). You must use the new rolls."

How do you calculate the increase in the average damage when you use this ability? It seems to me you would use this to only reroll dice that came up 3 or less however you have less of a chance of increasing your damage when you reroll a 3 than when you reroll a 1.

I know something of probability from playing tabletops for so long but figuring this out is beyond me.