Does it actually matter?
This is the question you should always be asking when something like this comes up. If it’s not directly relevant to the story, and the players are not trying to do something silly like carry around hundreds of sheets of paper, then it simply does not matter, and thus you shouldn’t waste time worrying about it.
For reference, when I run games, the usual turn-around point where I start worrying about weight of items with no listed weight is when a player is carrying around a stack of more than 10 of that item.
But really, the actual numbers...
The weight of papers, parchments, and vellums is typically measured using what is known as ‘grammage’, which is the mass of the paper per unit area, typically expressed as grams per square meter (typically abbreviated as ‘gsm’ in English-speaking parts of the world, though the correct SI unit is g/m²).
For parchment
Most parchment has a grammage of about 140-160 g/m². We’ll assume 150 g/m².
At 150 g/m², an 8.5 by 11 inch sheet of parchment would be just a hair over 9 grams, or approximately 0.32 ounces, which is roughly 50 sheets to a pound.
A metric A4 sheet is close enough to this to not matter.
In practice, the sheets of parchment are probably not 8.5 by 11 inches (or A4), as this is more just a modern standard than some practical requirement.
For paper
Modern ‘office’ paper has a grammage of about 80 g/m², but is lighter than what is likely available in most fantasy settings, so we’ll assume about 100 g/m² below.
At 100 g/m², an 8.5 by 11 inch sheet of paper would be just a tiny bit over 6 grams, or approximately 0.2 ounces, which is roughly 80 sheets to a pound.
Just like with parchment, an A4 sheet would be close enough to not matter, and the sheet size is probably not 8.5 by 11 or A4.