Is there a rule governing how I should factor in different damage types when a player uses a weapon to damage an object? For example, using a dagger or maul on rope rather than a scimitar, or using a long sword to gouge a hole in a sack.
Yes. See Chapter 8 of the DMG, the section entitled Objects. A partial quote:
When characters need to saw through ropes, shatter a window, or smash a vampire’s coffin, the only hard and fast rule is this: given enough time and the right tools, characters can destroy any destructible object. Use common sense when determining a character’s success at damaging an object. Can a fighter cut through a section of a stone wall with a sword? No, the sword is likely to break before the wall does.
The section then goes on to suggest ways of giving AC and HP to objects, as well as taking into account factors such as size, damage types, and thresholds. Less a hard-and-fast rule and more a collection of guidelines.
You asked, "Can a character use a dagger to cut a rope or only poke holes in it?"
Since the PHB defines the damage done with a dagger as piercing, that is an argument that daggers can't do slashing damage.
The same section of the DMG says:
Objects and Damage Types. Objects are immune to poison and psychic damage. You might decide that some damage types are more effective against a particular object or substance than others. For example, bludgeoning damage works well for smashing things but not for cutting through rope or leather. Paper or cloth objects might be vulnerable to fire and lightning damage. A pick can chip away stone but can’t effectively cut down a tree. As always, use your best judgment.
So while you could argue that daggers only do piercing damage, you could also reasonably note that common definitions of daggers note they have edges.
A dagger is a knife with a very sharp point and one or two sharp edges, typically designed or capable of being used as a thrusting or stabbing weapon.
To me that the DMG says things in this section like "use common sense", "use your best judgment", it is entirely reasonable to note that daggers have edges, therefore they can cut ropes.
Furthermore it is reasonable and entirely supported by the DMG to note that a maul isn't going to do much (direct) damage to a rope, while a sword has a pointy tip and can reasonably poke a hole in something, at least with some effort.
Presently, I use what makes sense in narrative. I rule that a scimitar does normal damage to a rope, but that a rope has resistance to bludgeoning. Similarly, the dagger does half damage.
The half damage isn't so much about the dagger not being able to cut the rope, but rather the scimitar is so much better at it.
But a spear is right out. Because that seems weird.
Unfortunately, I have no support for this other than it makes sense to me and our group accepts it.