True Polymorph transformation isn't actually permanent.
RAW it is still "a creature transformed by magic", so the permanent transformation still can be dispelled, through the Dispel Magic spell or an antimagic field. When you are permanently polymorphed, a creature with Truesight can still see your true form.
All these things indicate that a polymorphed creature still has its true (but somehow hidden) form. Another evidence is that Power Word Kill affects the true form, not the assumed one.
Being polymorphed into a yonger being doesn't automatically prevent your true form from aging. RAW imply it will age, because there is no rule that says your true form resides in stasis while you are polymorphed. However, rules don't clarify this point explicitly, so I suggest to figure it out by using the general RAI idea about aging.
Can D&D magic prevent aging?
Death by old age has special meaning in D&D magic. Even the True Resurrection spell won't work:
You touch a creature that has been dead for no longer than 200 years and that died for any reason except old age...
nor other spells:
You touch a dead creature that has been dead for no more than a century, that didn’t die of old age...
You touch a creature that has died within the last minute. That creature returns to life with 1 hit point. This spell can’t return to life a creature that has died of old age...
The only spell that can return you from dead in this case is the rare druidic Reincarnation spell, which doesn't return your original body, but gives you a new body (hence, a new true form) instead.
The only potion that actually can make your true form younger is the Potion of Longevity, but it has special condition - you can't drink it for ever, because you will age eventually (straight away, if you're unlucky):
Potion of longevity
When you drink this potion, your physical age is reduced by 1d6 + 6 years, to a minimum of 13 years. Each time you subsequently drink a potion of longevity, there is 10 percent cumulative chance that you instead age by 1d6 + 6 years.
If no magic can easily give you eternal life, why True Polymorph should be different?
It's like the picture of Dorian Gray
Dumbledore could actually try to prolong his life by all the magic means, inclding the True Polymorph method. How it will ends is up to the DM.
I can suggest the Dorian Gray scenario. Dumbledore assumes an elf form and lives for 200 years. Eventually he lose the assumed form. Maybe a powerful envier casts Dispel Magic on him. All the 200 years return to his original form, turning him into an incredible old man. He died from old age right at the moment.