You say:
"I'm trying to figure out if there's any good reason Tier 4 PCs wouldn't bring Adamantine Armor with them when using Astral Projection."
Probably not.
TLDR: Given a chance, those PCs for whom armor is appropriate would likely be fully kitted in it, or maybe carrying a set just in case. They might or might not have every bit of information regarding adamantine armor and its advantages and disadvantages compared to other choices, but they likely have every bit of information that they had the time and resources to gather. So, it's really a question of PC sleuthing, DMing, and world-building.
Tier 4 PCs are likely knowledgeable, powerful, wealthy, and resourceful
Tier 4 PCs (levels 17-20) are described as "masters of the world":
By 17th level, characters have superheroic capabilities, and their deeds and adventures are the stuff of legend. Ordinary people can hardly dream of such heights of power — or such terrible dangers.
In my experience, such characters will do everything in their power to face a significant danger with every tool that can be obtained in the given time and with the given resources.
Tier 4 PCs likely know of adamantine armor
Adamantine armor is merely uncommon. With the defaults presented in the DMG and other source books, Tier 4 PCs, masters of the world, would likely have broad knowledge of uncommon magic items, and so would most likely know of adamantine armor.
Characters with reason to do so, such as those who could wear such armor, given time and reason, would likely know details about it. Its properties. How it compares and contrasts to other expensive armor. Where to obtain.
You need to decide what level of knowledge they have, and is it automatic, a knowledge check, a conversation with an NPC, or a side quest, or potentially, just not available.
Tier 4 PCs likely know of the Astral Plane
Average inhabitants of the default setting might or might not know much about the Astral Plane. Likely their knowledge is rudimentary, apocryphal, fragmentary, and incorrect.
However, Tier 4 PCs are masters of the world, and by definition "traverse otherworldly realms and explore demiplanes and other extraplanar locales". They are likely to have broad knowledge of the planes of existence.
Characters with reason, such as those who could travel to other planes, or who have such connections, or who previously studied such things, would likely know details about the Astral Plane. It's properties. It's inhabitants and dangers.
You need to decide what level of knowledge they have, and is it automatic, a knowledge check, a conversation with an NPC, or a side quest, or potentially, just not available.
Putting it together
So, Tier 4 PCs are quite likely to have general knowledge of adamantine armor and the Astral Plane. How extensive that knowledge is, is up to you. And up to the PCs, for that matter. Given time and resources, they can likely find out most detail available. They are likely to then equip themselves as best as possible.
Your question has the title of "Do characters understand the concept of a 'Critical hit'?"
Yes. They are us, and we are them.
Do they know of natural 20s? Do they conceive of themselves as characters being played by creatures of a higher reality? Whew, you could play it that way. It is a stable of fiction that we mortals are merely pawns of a bigger game played amongst the beings of a higher order of existence.
The rules are quite muddled on the difference between player and character. For instance, the rules routinely change perspective, sometimes talking about players, sometimes talking about characters. From the introduction to the Basic Rules, "How to Play":
Sometimes one player speaks for the whole party, saying, “We’ll take the east door,” for example.
Really . . . it's the character talking about the east door. Or is it?
But the character just needs to know some hits are extraordinary
The player knows a 20 on a die is a critical. The character knows some attacks are better than others.
Conclusion
And any character who's had a chance to prepare, will know that going into the Astral Plane involves significant risk to that precious silver cord, and may will choose to try to mitigate that risk though any way possible, including guarding against the skillful or lucky hits that might sever the silver cord.
I know I would. I mean, my character would.