I'm new to D&D, and I'm not getting what base stats do for your character during play. It seems as if the only thing that really matters during an adventure are the modifiers that you get to add to your dice rolls. Are your base stats used for anything during an adventure?
Lots of things!
The following list addresses obscure corner cases which require you knowing the character's ability scores but are negligible in 90% of games played.
The Strength score determines how many pounds of weight your character can lift, carry, or push.(Using Ability Scores, PHB 176)
It also determines how far and how high your character can jump. (Special types of movement, PHB 182)
The character's Strength score is also used to know if your character's speed suffers from wearing heavy armor. (Armor and Shields:Heavy Armor, PHB 144)
Determines how long your character can withstand being immersed in frigid water. (Wilderness Hazards, DMG 110)
The Intelligence score determines how fast you can move in the Astral Plane. (Astral Plane, DMG 47)
Charisma Score is used to track the Loyalty of NPCs to you or your party. (Loyalty Score, DMG 93)
If your DM allows multiclassing, Ability Scores determine what you can and cannot multiclass into. (Multiclassing Pre-requisites, PHB 163)
If your DM allows Feats, some of these Feats also require a minimum ability score. (Feats, PHB 165)
But perhaps the most important use of keeping track of an Ability Score is to determine what score you may want to increase as you level up, especially with odd numbered scores.
In summary, you are correct that you would need to use your Ability Modifiers most of the time, but writing down the score doesn't hurt; you get used to looking at a score and telling the modifier anyway.