A few years ago, I played a D&D 3.5 game in Forgotten Realms with an evil party. The characters (played by my brother and me) were sociopaths, bent on exploiting anyone and everyone. The game revolved around our characters confronting a situation and then coming up with a plan to exploit it as much as possible.
This campaign gave me some of the best memories of role-playing to this day. So, when it was my turn to run a game, I decided to recreate the feeling, only this time the whole game would be designed around it.
I set it in a modern-day urban environment. The PCs were a bunch of orphaned teenage brothers with antisocial personality disorder. Each session was designed to revolve around a single "bet": the characters would have to exploit a situation, looking for gaps in the structure of society and human psychology. I used a mostly free-form skill system without rolls or ranks. If you are smart, it means you can reason about the most difficult situations. If you are charismatic, it means everyone except the most cynical people love you. If you are good with computers, hacking most government systems is a piece of cake.
Still, it didn't turn out as I had hoped. Most of the time, the players couldn't come up with creative plans and even when they did, it was nothing to celebrate about.
So, I changed the system. I gave the players more power over what happens beyond their characters. Scenes, puppet characters, narration. But things didn't improve.
So, what do you think went wrong? How would you design a game around this principle?