The description of the polymorph spell (BR, p. 266) says:
The target's game statistics, including mental ability scores, are replaced by the statistics of the chosen beast. It retains its alignment and personality.
The Ammunition section of the rules on monsters (MM, p. 12; BR, p. 114) says:
A monster carries enough ammunition to make its ranged attacks. You can assume that a monster has 2d4 pieces of ammunition for a thrown weapon attack, and 2d10 pieces of ammunition for a projectile weapon such as a bow or crossbow.
The Giant Ape statblock (MM, p. 323) includes the Rock attack:
Rock. Ranged Weapon Attack: +9 to hit, range 50/100 ft., one target. Hit: 30 (7d6 + 6) bludgeoning damage.
If a creature's game statistics are replaced by the statistics of the chosen beast when polymorphed, does a character polymorphed into a Giant Ape carry rocks on him to make its rock attack once transformed?