The caster continues casting the spell.
Taking the same quotes from @Xirema's answer, we have the following for casting spells with a long casting time:
Certain spells (including spells cast as rituals) require more time to
cast: minutes or even hours. When you cast a spell with a casting time
longer than a single action or reaction, you must spend your action
each turn casting the spell, and you must maintain your concentration
while you do so (see "Concentration" below). If your concentration is
broken, the spell fails, but you don't expend a spell slot. If you
want to try casting the spell again, you must start over.
—Casting Time, Player's Handbook, pg. 202
And the surprised condition:
If you're surprised, you can't move or take an action on your first
turn of the combat, and you can't take a reaction until that turn
ends. A member of a group can be surprised even if the other members
—Surprise, Player's Handbook, pg. 189
While at first glance it would appear that the casting would end, I would argue that there is difference between spending and taking an action.
Spending vs taking
Both terms are used heavily in the rules, but are never clearly defined beyond their standard English meanings.
However, the words do seem to be consistently used in different ways.
Spending is used to represent the consumption of a resource.
Taking is used to represent the choice to consume a resource.
A turned creature must spend its turns trying to move as far away from
you as it can, and it can’t willingly move to a space within 30 feet
of you. It also can’t take reactions. For its action, it can use only
the Dash action or try to escape from an effect that prevents it from
moving. If there’s nowhere to move, the creature can use the Dodge
-- Turn Undead, v1.0 Basic rules PDF, pg. 24
An undead has a turn as a resource, and it must consume it by trying to move away.
It also cannot take a reaction. Why didn't the designers say it can not spend a reaction?
We can't read their minds, but I found many other examples where spend is consistently used to refer to consumption of a resource, whereas take is used to refer to a choice to consume a resource. (Look at class abilities for a lot of "take" examples.)
I would argue that, by RAW, a creature that is casting a spell with a long casting time has already made the choice before being surprised to be casting that spell. As such, they can still have their action consumed by that spell casting when they are surprised. They simply cannot make any choice to have their action consumed in any other way.
This also fits very nicely with what makes sense: If a caster is surprised, they don't really understand that combat is going on, and so would continue to keep concentrating on their spell casting.
I admit it is not perfectly consistent in how the two words are used in the rules, but given that it makes more sense AND is a reasonable reading of the rules, I would definitely say that allowing the caster to continue casting the spell is RAW.
Some comments bring up the rules of concentration as an overlooked part of this answer. I do not consider them to be at the core to the question though. The question quotes the following statement:
you must spend your action each turn casting the spell
The word "must" tell us that this is a requirement that must be met, although it doesn't explicitly state what happens if it fails. I believe that the fact that the rules immediately follow with
and you must maintain your concentration while you do so (see
“Concentration” below). If your concentration is broken, the spell
fails, but you don’t expend a spell slot.
means that the designers considered failing concentration to be the only way they expected a long casting time to be interrupted, and so did not bother to explicitly say that failing to spend an action was something that had a failure state.
The conditions that may cause you to lose concentration are as follows (Basic Rules PDF v1.0 pg 84):
- Casting another spell that requires concentration.
- Taking damage.
- Being incapacitated or killed.
None of those are triggered by the surprised condition, and so concentration is not broken. However, the question is not asking about whether concentration is broken.
The "must spend your action" statement is what was in question, and so is what my and Xirema's answers are focusing on.
As mentioned above though, I do take the lack of direction on what happens if a creature does not spend an action each turn as further indication that the designers did not intend for the "must spend your action" statement to be something that can ever be involuntarily failed.
As such, despite it admittedly being a bit hair splitting, I believe there is a difference between taking and spending an action, and that an action is still spent while a creature has the surprised condition.