More Elegance?
I read nitsua60's answer (which was the accepted answer at the time) and found that it's not very elegant to use the location of the dice. It's not always visible in a single glance, and there is the possibily of people disagreeing on this. I'd prefer something that you can see in a glance or two, so no rerolling and no using the position of where the dice end up.
That's the reason I came up with a different system that meets those goals at the cost of being less close to actually being linear. This solution does not adhere to your example percentages, nor is it actually linear, but on the scale from 1 to 5 dice, I think it'll feel close enough to linear.
A red die
So, one of the dice is different from the others. The easiest is if it's a different color. We'll call this the red die. You always roll the red die. That means that if you roll one die, it'll be the red die and if you roll more than one die, it'll be the red die and a number of other dice.
Now, if the red die comes up a 1 and none of the dice comes up a 6, it's a fumble. The probability of fumbling will be as follows:
Number of Dice |
Probability of Fumble |
1 |
16.67% |
2 |
13.89% |
3 |
11.57% |
4 |
9.65% |
5 |
8.04% |
Note that adding a second die decreases the chance by approximately 2.8 percentage points, while adding the fifth die will decrease the chance by approximately 1.6 percentage points. This is not quite linear, but I do feel that it's close enough to feel more or less linear.
Side effects
Originally, this answer had a solution that was "a red 1 and no other ones", but based on a suggestion in the comments by both @Nick543211 and @NathanHinchey I changed it to "a red 1 and no sixes". This has the same probability distribution, but seems to have fewer side effects. For example, the original system had rolls that feel like they should have been fumbles (like three ones) but weren't.
Another side effect is that this changes the probability distribution of success and failure rates somewhat. There may be some rolls that would have been a success without the proposed fumble system, but now end up being a fumble. For example - if we assume that the dice are simply added up and then compared to the target number - you could roll two fives and a red one when you're up against a target of 10. The change has made this problem occur far less and I'd say it's not that of a problem anymore. You could still try to balance it with a "critical success" system if you do feel it's a problem.