First, decide if you should even be using a game mechanic for this.
In the large majority of cases of an object being on fire, you probably shouldn't be using these kind of rules anyway.
Most of the time, setting objects afire should just be background description, with maybe a potential tactic of shoving somebody into the fire for damage. If the room has caught fire from fire spells or some such thing, it's just a setting. We don't need to decide how many rounds it takes for thus-and-such item to burn.
But assuming an object of importance is on fire, it's a narrative question, not a mechanical one. If there's a book that the PCs need to save from burning, I'm going to figure out how many rounds I want them to have before the book is ruined, and that's the answer. I don't need to figure out HP vs. damage and then calculate the resulting number of rounds. If a rope bridge has caught fire, the question at hand is "How much time pressure I want the party to be under?", not "How many hit points does a bridge have?"
If it matters, just pick a die.
In the rare case that the PCs are intentionally setting something on fire as part of an ongoing attempt to break it -- such as, say, the fighter is hacking at the gate with his axe, the druid sets it on fire, and time is important so it's important to know how much the fire contributes -- I'd probably just decide on a die to apply, and that's that.
Alchemist's fire burns for 1d4 each round, while a puddle of burning oil deals a flat 5 damage each round (which is roughly 1d10), so there's plenty of variability already -- I'd just pick something in that range and apply it. A d6 each round sounds about right to me off-hand, and apply vulnerability if the item seems particularly easy to burn.
But that kind of scenario will almost never happen at the table, so I'm actually not all that concerned about figuring it out ahead of time. I'll just use my judgement when and if it actually comes up.
Is it realistic? Maybe not. Even a modest pine-board door would take many minutes to burn away in real life. But D&D is more about the heroic, dramatic, and narratively interesting than it's about the realistic, so I don't expect my players to start complaining that wooden objects burn up too fast in combat. They don't want to argue about the physics of combustion, they want to be on the other side of the door!