Reading through the new rules for Sidekicks in Tasha's Cauldron of Everything (p. 142), it seems like they must roll for HP when they level up. Am I understanding this right? Or can I use Monster Manual rules for average?
There are no rules saying a sidekick can take average on Hit Dice.
Every player class in the Player's Handbook (and the Artificer class in both Eberron: Rising from the Last War and Tasha's Cauldron of Everything) states that a player can take the average on each Hit Dice they gain when leveling up, as well as stating that the 1st level in the class is not a roll of the Hit Die, but the whole amount.
However, according to what's stated in the sidekick rules on page 142 of TCoE, you would need to roll for each new Hit Die the sidekick gains.
Whenever the sidekick gains a level, it gains one Hit Die, and its hit point maximum increases. To determine the amount of the increase, roll the Hit Die (the type of die appears in the sidekick's stat block), and add its Constitution modifier. It gains a minimum of 1 hit point per level.
Of course, a DM could rule otherwise, allowing the sidekick to follow the rules for Hit Points average presented for monsters in the Monster Manual (page 7), or they could even rule that said sidekick could use the average of player classes' Hit Dice.