TLDR:
Roll regular advantage, then also roll a d4, if d4 rolls 1 then flip the value (20->1, 1->20, 19->2, 11->10, etc)
Inpired by the answers I saw...
So after I posted the question I continued experimenting and did manage to come to the realization that to get the distribution I was looking for, the simplest way was probably just to modify the advantage roll some of the time.
After seeing the simplicity of Eric's answer I was flabbergasted how I could miss the obvious, and I put away what I was working on. However, after all the creative answers and seeing that they all have benefits in specific situations (eg: what if you don't have an easy way to distinguish the dice, or if using an online roller, or to avoid rolling extra dice), I was inspired to continue and see if I was close to something, and I managed to create another strategy...
It's not as elegant as all those shared here, it requires rolling another dice, and it even involves some simple math... but at least it has the right distribution, and the procedure might even provide a fun suspense/drama to the game.
Strategy
What I came up with is a minimally complicated way to dampen the distribution around the middle. The way I'll demonstrate works like Erik's answer in terms of probability. It's easy enough to perform, only involves 1 extra dice, the math is minimal (but that is more to complicate it). And what's great about this is you can easily adjust the flattening effect. The mechanic is to roll a "flip" die.
Let me explain what I mean by "flip"...
To flatten the probabilities, we want to balance around the middle (10.5). If some low percentage of the high values became the corresponding low value (reflected across the middle) it would flatten a bit. And we can apply this flip to both high and low values because there are less of the low values to flip. So when we flip: 20 becomes 1, 1 becomes 20, 19 becomes 2, 2 becomes 19, 18 becomes 3, 3 becomes 18, and so on until 11 becomes 10, and 10 becomes 11. The calculation I'm using is { M=10.5, X+(M-X)*2 }. We are "flipping" the number over the mid point. We do this at a set probability, say 1 in 4 (that's rolling a 4 sided dice and avoiding a 1, or hoping for a 4, depending on how you want to run it).
As for the math, it's not hard if you think about the number bonds for 10 (first bit of number sense people learn: 1+9, 2+8, 3+7, etc), because the flip value is just that plus 1. I don't think that's hard (then again I like math), but it will definitely put off some people.
Here are a couple examples
Conceptualizing the flip dice as "keeping" a good roll (just don't roll 1):
- Roll with advantage ( 18 & 4 becomes 18), we like this roll...
- Now roll the flip die to avoid the flip, a d4 (2)
- Our flip die did not fail us and we keep our 18!
Conceptualizing the flip dice as "saving" a bad roll (get the crit):
- Roll with advantage ( 1 & 6 becomes 6), we still have a chance...
- Now roll the flip die to get that flip, a d4 (4)
- Our flip die came through! That 4 becomes 17!
More Thoughts
The procedure could add good drama (d4 is dramatic either way), but also has the opportunity to be anticlimactic (rolling a 10 or 11, and generally close to the middle), but the statistics work out (AFAIK, and this is the only way to accomplish this so easily that I've found yet). Using a d4 as the flip die is perfect for "half" advantage, but you could use a d6 to be much closer to regular advantage with just a bit of flattening but still a rather significant addition of risk that shifts the chance of a crit fail from 1-in-400 to near 1-in-50. And of course you can play with it further, for instance a flip on 1 OR 6 of a d6 withh result in a 1/3 chance of a flip and a much flatter advantage closer to a regular d20.
Here's an anydice implementation of this method: https://anydice.com/program/2492e
Flexibility
What's interesting about this technique is it seems to work over almost any other roll. Anything you want to dampen, just use a flip dice to flatten how much you want. Only want a little bit of dampening: use a d12. Want to flatten it by half: d4. I'm still playing with it, but I like that it can be customized for any roll.
Here's a version that illustrates it's flexibility: https://anydice.com/program/24936
Here's an image of dampening 2@4d20 (roll 4d20, drop high and low, use remaining 2 for advantage... aka take the second highest number)

Though keep in mind this isn't actually dampening towards equal probability, it's bringing the reflections closer. So for a curve that is symmetrical around the middle it will do nothing. So the "sometimes apply a normal roll" is better at a true "dampening" effect. This method remove bias towards one end, but not bias towards the ends/middle.