More Elegance?
I read nitsua60's answer (which was the accepted answer at the time) and found that it's not very elegant to use the location of the dice. I'd prefer something that's simple, doesn't require rolling and doesn't require the extra bookkeeping of things like where the dice end up on the table. Therefor, I came up with something that meets those goals at the cost of being less precise. This solution does not adhere to your example percentages, nor is it actually linear, but on the scale from 1 to 5 dice, I think it'll feel close enough to linear.
A red die
So, one of the dice is different from the others. The easiest is if it's a different color. We'll call this the red die. You always roll the red die. That means that if you roll one die, it'll be the red die and if you roll more than one die, it'll be the red die and a number of other dice.
Now, if the red die comes up a 1 and none of the dice comes up a 6, it's a fumble. The probability of fumbling will be as follows:
Number of Dice |
Probability of Fumble |
1 |
16.67% |
2 |
13.89% |
3 |
11.57% |
4 |
9.65% |
5 |
8.04% |
Note that adding a second die decreases the chance by approximately 2.8 percentage points, while adding the fifth die will decrease the chance by approximately 1.6 percentage points. This is not quite linear, but I do feel that it's close enough to feel more or less linear.
Side effects
Originally, this answer had a solution that was "a red 1 and no other ones", but based on a suggestion in the comments by both @Nick543211 and @NathanHinchey I changed it to "a red 1 and no sixes". This has the same probability distribution, but seems to have fewer side effects. For example, the original system had rolls that feel like they should have been fumbles (like three ones) but weren't.
Another side effect is that this changes the probability distribution of success and failure rates somewhat. There may be some rolls that would have been a success without the proposed fumble system, but now end up being a fumble. For example - if we assume that the dice are simply added up and then compared to the target number - you could roll two fives and a red one when you're up against a target of 10. The change has made this problem occur far less and I'd say it's not that of a problem anymore. You could still try to balance it with a "critical success" system if you do feel it's a problem.