It is best not to give any extra damage
First, Pushing Attack is already part of an attack which does damage, and the maneuver already adds extra damage to the maneuver by adding the combat superiority die to it:
When you hit a creature with a weapon attack, you can expend one superiority die to attempt to drive the target back. You add the superiority die to the attack's damage roll [...] On a failed save you push the target up to 15 feet away from you.
So this maneuver already includes damage coming from the maneuver itself which presumably models a variety of different things that can happen during the maneuver including the force of the push itself. Adding damage to this for pushing a creature against a wall or barrier seems redundant.
Rewarding clever play is one thing, but there are almost always going to be walls and enemies to push into them so this isn't just going to be a one-off addition. You are effectively giving the player a very consistent way to add damage to an ability that wasn't intended to do that damage. So, you need to consider how you are affecting the balance of this feature for the entire rest of the campaign.
It is worth noting also that anybody can shove instead of taking an attack, this means that, if you follow the logic in your ruling to its logical endpoint, your ruling should apply to those players that don't have the feature as well meaning that you would now be adding damage to a move that didn't have any damage added to it at all.
It is up to you how much you want to affect the balance of the ability
Again, I don't think you should, but if you have your heart set on trying this out, some considerations:
In the end, you are making this ability more powerful than the designers intended. Only you can decide what level of power you are comfortable adding to the ability.
By adding damage to the ability you are going to make the ability more powerful and make the balance change. The less damage you add, the less of a balance change it will be. Starting with a d4 or low amount of static damage (eg 2) should be your first step in investigating this. This would be on par damage-wise with giving a free unarmed attack with every successful shove into a wall.
A good thing about adding static damage is that it will not get multiplied on a critical hit which means that it becomes slightly less of a balance concern than if you rolled dice.