Yes, it's fine to make it up as you go along. You'll get better at it the more you do it. I have two bits of advice.
First, prepare in as modular a manner as you can. Prepare rooms more than dungeons, characters more than residents. That way if your PCs pass through the Catacombs of Despair without finding that one secret room, you can throw it into the Cliffs of Despair without anyone the wiser; that encounter with Prince Bungholio can occur two towns over if you just preparing that he has A favourite tavern instead of prepping one specific tavern to find him in.
But the real gem in that is so you can use the ideas your players give you more easily! "Boy, we keep finding these Orc Shamans, I wonder if it's not some cult!" they say... So you turn the Necromancer in the Catacombs into an orc and suddenly the undead army they have to stop have become nothing more than the proverbial "beginning of the end," mere shock troops for an orc horde! And you're ready for that, because you didn't lay it all down mentally in stone.
The second advice is weird... Set the game up as not what really happened, but as one of the characters recounting their glory days in a tavern as an old, retired adventurer. This can mentally help cover mistakes, and you can even let that player (rotate each time) explain what really happened when mistakes come to light. This lets your players join in creating the narrative!