Talk to your players
Just...step out of character for a moment and talk to them. There's no rule that says the DM can't help players remember the features their characters have. If you're more comfortable doing it outside of the game, then chat with them in private (in person, text, chat, whatever).
If you were trying to teach someone how to play checkers and they clearly forgot that multi-jumps are possible...if you want them to learn how to play, you remind them. There's no rule that says that once the game has begun, everyone must muddle through on their own.
Learning D&D is a process...and knowing how much to help with that process depends on your players. As a DM who has run for a lot of new players, I routinely have to remind my players of all the things their characters can do. Because these are things the character should know but the player does not.
Your concern that this strips away player agency is not something you need to worry about. You are not mandating that they do something, you are not taking away their choices. You are simply pointing out that another option exists.
So, here is what I recommend.
Ask them if they'd like input from you.
Go with something like...
Hey guys, I know you're all new to D&D, so I wanted to ask you something. I've noticed there are times when you guys forget about some of the stuff your characters can do, or you get stuck on something that has an alternate solution. I know D&D can be a lot to handle all at once, would you like me to help remind you?
If they say yes, a simple "Hey, just in case you forgot, your character can cast Mage Hand...which is kind of like short-range telekinesis." or "Hey, just a bit of advice...when you see a bunch of enemies in a clump like that, hitting them with an area of effect spell, like fireball, is usually a good idea."
For more complicated things (such as breaking and entering), I will call for a check that provides a hint right in the call. To give a recent example, the party was looking at this well-secured building, trying to figure out how to get in. So, I said to the party Rogue...
Give me an Investigation check to case the building, see what your options are for breaking in.
The rogue knows how to break into a building. There's no need for a check to see if they remember how to make locked doors stop being in their way. Instead, I called for a more specific check to determine how good of a job the Rogue did at picking out the specific vulnerabilities of this building.
Even if their check wasn't great...that still plants the idea in the party's head of "what other ways can we break in?"
Checks should be for things that challenge the character, NOT the player
What I would NOT suggest is calling for rolls for your characters to remember what they can do. Just tell them. This is not something their character has to try to remember or struggle to recall. In the example of Mage Hand...that character has cast that spell a vast number of times during their training...remembering that they can cast that spell is not 'challenging' for that character, it's hard for the player. Dice rolls should be about the character, not the Player behind them.
D&D isn't a competitive game...the DM is yet another person who is working with the party in order to make an engaging, fun game. The fact that they run 'the enemy' doesn't matter. You aren't their foe, you aren't trying to beat them. If their lack of experience is getting in the way of them having a good time, then volunteer to help!