A spell's duration begins whenever the spell's effects begin
Here's how the rules define spell duration:
A spell's duration is the length of time the spell persists.
This length of time starts when the spell's effects start and ends when the spell's effects end (though other effects can cause the spell to end early, such as Dispel Magic or dropping concentration). There are a few exceptions to this, such as Levitate and Haste, which both explicitly specify effects that occur after the spell ends. However, the above is the general rule. (Additionally, as far as I know there are no spells that take effect before the casting is complete, with the exception of spells that explicitly or implicitly manipulate time.)
In most cases, a spell's effects (and thus its duration) begin immediately as soon as the casting is complete. One exception to this is a readied spell, as you have noted. In the case of a readied spell, the spell's effects do not begin until some time after the completion of the casting, when the specified trigger occurs and the caster uses their reaction to release the spell's energy. Thus, a readied spell's duration only begins when the spell's energy is released and the spell takes effect, not when the casting is completed.
Because readying a spell separates the casting and the start of the spell's duration, you'll need to carefully read the text of any game feature that interacts with spellcasting. For example, if an effect is triggered "when you cast a spell", in general that effect triggers on your turn when you use your action to cast the spell, even if you are readying the spell to release later. (Although note that if such a feature modifies the effects of the spell when it takes effect, the exact timing of when the effect occurs may turn out to be irrelevant after all.)
To take a specific example, Invisibility "ends for a target that attacks or casts a spell." In theory, you could ready a spell that doesn't involve an attack roll, then have someone cast Invisibility on you, and finally release the readied spell while invisible, and you would still be invisible after releasing it, because you never cast a spell while you were invisible.
One other weird edge case is that, if you ready a spell, it's possible to cast the spell but then never have its duration or effects begin. This can happen if the trigger never occurs before your next turn or if your concentration is broken before you release the spell.
