You ask about the effect:
What would the consequences be of removing Common from a large
majority of creatures?
It often makes the game needlessly difficult
We - the people I played with from about 1975 to about 1987 - tried this in early versions of the game1, in a variety of ways, and it simply slowed down play with little to no other benefit unless Common was indeed a very common means for communication in a game where role-playing, which has a heavy communication element, matters.
I am even running into this problem in a published adventure in 5e where a key NPC does not speak common, and only one of our five PCs speaks Elvish. We had to do this awkward translation thing: "OK, so he says to him {content}" and then that PC has to tell everyone else.
I finally decided "Heck with it, this NPC speaks Common!" (and the PC version of sea elves speak common per DDB) so that we could Play This Game - which is far more of a fun game when you aren't slowing it down with waiting for one PC to translate for all of the others.
Languages differences can make for great and rewarding play
A few years ago we, the players, had a challenge: none of us spoke goblin and we were in the jungles of Chult. The Batiri did not speak Common . So, for this Single encounter, we had to arrive at a novel way to communicate with them, and we did. The Bard kept casting Minor Illusion to show pictures of what we were trying to convey, and, he had already cast Comprehend languages so that he understood their response. The other party members described their sign language efforts.
This was great for an occasional challenge; it's lousy for a session after session, encounter after encounter, obstacle to communicating with NPCs and other denizens of the imagined world. Role playing games are built on being able to communicate with one another at the table.
Depending on who you have at the table, using a variety of languages to inform a style of game play can be fun; but the appeal of that style varies widely. (Thanks @GMJoe for reminding me of that Q&A). This is a style that you need to discuss and try out with your players. If you all like it, then have fun with it. It may take a few sessions to see what other adaptations your table wants to make.
Having Common as an almost universal lingua franca makes the game play more smoothly.
Having NPCs and monsters not speak Common makes for an interesting exception now and again, but does not make for an interesting standard situation.
You want languages to matter more?
OK, do that, but I suggest that you pick your spots. Or, provide some means, via spell or magic item that perhaps have a cost or an opportunity cost, that allows most interactions to be smoothly executed at the table.
One possible silver lining
If you choose to make languages and language proficiency matter more, it gives you a way to spread the spotlight around depending on who speaks a given language that is needed for a given situation.
1 D&D is the game I refer to. PF is a spawn of D&D 3.5e which has carried over many of the conventions in the game's DNA: Common being very common is one such.