Here is a thing that I have done in the past, when I worried about combat difficulty: I offered the party a free do-over.
Several times, when it looked like the party was in trouble, I've said: "guys, I think I misplayed this battle. I ruled that X monster would do Y thing, but that was unfair -- really it should have done Z thing instead. Would you like to just play the battle over?" Then we rewound, and they fought the battle again with better tactics.
One time, I knew that the group was heading for a tough battle, so I told them: "this weather phenomenon you're seeing is the Storm of Foresight, a gift from your goddess. If you cast a spell slot into it, you can see the future." The cleric agreed to use a spell slot on it. Partway through the session, I turned to the cleric and said "...and this is the future you saw, when you gazed into the storm. Do you accept this future? Or would you like to change it?" She told me she'd like to change it, so we did the battle again.
Once, I told the players: "Guys, this curse you got hit with was a little bit railroady. The plot of the campaign is about getting rid of the curse, so I sort of had to make sure the curse happened. But I want to make it up to you, so here's what I'll do: I'll give you one do-over. One time during the campaign, you can tell me you don't like the outcome of a decision, and we'll rewind and do it over."
All of these conversations went pretty well -- the players accepted what I did, and they didn't seem to lose belief in the game world. That last conversation was my favorite, because it eliminated some tension. The group would be agonizing over whether to fight or flee, and someone would say: "guys, we have a do-over. Let's make a plan, do our best, and if it doesn't work we'll take the do-over and run." They never actually used the do-over.
I think this would work pretty well for you. You've told us how the group is learning, and you've told us about the terrible tactics they used. Offer the group the chance to do the battle over, without the knight -- maybe make up some reason why the first result was invalid, or tell them that the knight was needed elsewhere and pulling him into this battle meant bad consequences for the castle, or maybe just tell them you think it'll improve the game. It sounds like the group would do much better on a second run through.