I've had good experience with a system somewhat like this, which like Magician's suggestion focusses more on letting the players reward each other, but had as an extra criteria that you had to explain why you were awarding it. This turns out to be the key point.
The basic gist was that each person at the table (including the DM) would get one token (which can be a dice of XP for you, or an action point, or whatever you find enough to be a minor motivator). Then either during the game or at the end of it, any person could donate their token to another person when something they did something really cool. But they had to explain why they thought it was cool. This was especially relevant when someone saved it until the end, as they had to bring some fun part of the session back up.
I've found that most players were rewarded more by the words of encouragement than the actual reward itself. While many players might not openly tell you they enjoy being praised by their peers (a social faux-pas, I guess) it really does work. Adding the small mechanical bonus allows you to circumvent the awkward "I'm awarding you one compliment for being awesome" in favor of the easier "This is a mechanical bonus, and this is a compliment to explain why I'm giving it to you."
It's also very useful to the DM as it shows you which parts of the game the players value the most. The token could be given for essentially anything as long as the giver of the token found that it made his night more fun. Because you play with these people more often you not only learn what kind of things they enjoy (because they give an award for it) but you will also find your other players will start playing to entertain the other people at the table to try and claim the reward/praise from them.
This is also why it's important to give every person the same amount, otherwise people will start playing favorites to people who have more tokens to give. Which is usually the DM.
Most commonly the token (and explanation/compliment) were awarded for great in character jokes, brilliant strategies played out in character, but also occasionally for rolling well with failure, doing sub-optimal but totally roleplaying valid actions and things that simply make everyone at the table go "wow".
As an example, I once collected multiple tokens in the first half hour of the session because our party was ambushed, and my characters' first action was to lift his entire backpack overhead and then fire off the artefact cannon inside into the guy in front of him, blowing out the entire underside of the backpack and scattering a flaming shell and all his burning belongings all over the ambushers. Simply because it was the kind of impulsive thing he would do and he did not want to waste the time to actually "draw" the weapon first.
Because of the reward tokens, that moment received some extra spotlight as everyone had to say why it was cool and give over the tokens. It also upped the bar for the remaining tokens a bit of course, since fewer were left and people would not give them out lightly after such a start.
Receiving a few compliments (and some XP) about how it's great that my character would ruin a lot of his (not very relevant, mechanically) stuff in order to display just how itching he was to get in the fight, really helped me realise that it was one of the things the other players enjoyed about the character. It motivated me to roleplay it more, mostly because of the kind words. I don't even remember what I used the XP for, but I still remember the story even though it's been at least 6 years, because of the reactions I got from the table.