The Books are Silent
As far as I can see, the only rules for construction costs are on pg. 128 of the DMG. They detail the cosat for building various 'stronghold' structures. Nowhere does it tell you how much individual structural components of buildings cost.
Things to Consider
Given that the books say nothing, we will have to use common sense to determine a reasonable cost. The first thing to note is that this is a fine. The purpose of a fine was, and still is, to discourage people from committing the crime. Thus a fine usually charges much more than the stolen or damaged item was actually worth. I take an extract from this website I found with a quick Google search:
The thievery code in the Dalarna law called for a fine of three marks for pilfering one öre (a mark was worth eight öre).
As you can see, the fine for stealing 1 öre is 24 öre - a marked increase.
The next thing to determine is whether the property they damaged was that of a peasant or that of a nobleman (or other wealthy person). If it was a peasant's home, then it is safe to assume that they built it themself. If it was a nobleman's home, then he probably paid to have it built. For this reason, there would probably be a greater fine for damaging a nobleman's home than for damaging a peasant's home (a peasant would also be less able to press charges).
You also have to consider the character's ability to pay such a fine. Low level characters will not be able to pay a fine in the thousands of gold. Court systems today take the criminal's ability to pay into account. Maybe your court does too?
You might also want to consider whether a fine is really what you want. Other medieval punishments included:
- Time in the stocks (a pop culture favourite)
- Cutting off a hand or ear
One of those, or another inventive punishment based on your setting might be more interesting than a fine.
A Reasonable Cost
Let's look at the price of things in 5e. You can buy a chicken for 2cp (yes, that's copper pieces). A chicken was a valuable animal to have in medieval times. It would consistently lay eggs for food. If you bought a few and bred them, you could have a consistent supply of meat. A goat is 1gp (milk), a sheep 2gp (meat and wool), a pig 3gp (meat), a cow 10gp (meat and milk). All of these very valuable animals are extremely cheap by the player's standards (how many chickens could your players right now?).
A door is much less valuable than a cow. Jack wouldn't have got the magic beans for selling a door now would he?
So where does that leave us? We're looking at a fine of probably 1gp or less per door. The problem, of course, is that that sum is miniscule. You players will pay it without batting an eyelid. So what you need to do, is set a much much higher fine, not based on the value of the property destroyed, but on how much you think you need to charge to discourage this behaviour.
A Personal Experience
Once, my players decided to take some planks from a peasant's house. I can't remember exactly what they were for, but I am sure that the player's were well intentioned. Regardless, I decided that the town authorities would not stand for such vandalism, so I charged them a fine of (something like) 150gp. 150gp is an absurd fine for a few planks. It would have paid for the planks over a hundred times over. But that was not the point. I knew that my players could pay the fine, but also that it would discourage further such behaviour.
Don't think about the fine in terms of how much the property they destroyed cost. Instead, think how much you will need to charge them to make sure that they do not do it again, because that is what fines are ultimately for.