There are two aspects to this question:
Why have material components that are free or inexpensive?
Why make them all different?
Why have Inexpensive material components
Most of the time, the presence of a free material component will just mean the caster needs to interact with an arcane focus or component pouch. They will need a hand free to do this.
There are spells that require no material component. These may be especially useful, for example, for a ranger with the War Casting feat, wielding a sword and shield or dual-wielding — he has no need to unwield a weapon or shield and then handle his component pouch (note, rangers do not get arcane foci) to cast such a spell.
Spells that lack one component or another can be handy when some game condition or other deprives them of the ability to use the component (that being silenced, bound, or deprived of a component pouch/arcane focus.)
Why have material components be unique, individual items?
While a few somatic or verbal components are specified in spell descriptions (notably, Burning Hands requires touching one’s thumbs together) material components are specified in each spell’s stat block. Why?
Eye of newt, and toe of frog
One reason why this D&D tradition has survived (or been revived) is the same reason material components appear in Shakespeare: they do a good job setting a mood.
On the other hand, I think we could agree that calling out every syllable of every verbal component or gesture of every somatic one would simply be tedious.
Scrounging for components
If a caster doesn’t have access to a component pouch or arcane focus (generally because it was stolen or confiscated, but there might be other reasons) then individual components can play a role in game play. Some will be trivial to acquire, others difficult.
As such, they’re a little like encumbrance rules: some groups will never use them, many gloss over them most of the time — but they are there in case the situation calls for them. For example, Umbranus shares this memorable escapade:
In one game with an earlier edition my char could once save the party
after we were caught and tied up because I remembered how easy it is
to get the components for unseen servant. Back then it had no somatic
components but material component was similar to today: Piece of wood
and string. So I worked my clothes to pull a string out and scratched
the wood pole I was tied to until a piece came loose.
components this situation would not have been as noteworthy. And I
wouldn't still remember it.