In the DMG, Ammunition +1, +2, or +3 refers to a single piece of Ammunition
You have a bonus to attack and damage rolls made with this piece of magic ammunition
So strict RAW, yes, it would be prophibitively expensive.
Core Rules
Xanathar's Guide contains optional rules. Based on the core rules, this would be even more expensive (p. 129, DMG).
An item has a creation cost specified in the Crafting Magic Items table (half that cost for a consumable, such as a potion or scroll).
Item Rarity: Rare
Creation Cost: 5,000 gp
And it would take even longer
A character engaged in the crafting of a magic item makes progress in 25 gp increments, spending that amount for each day of work until the total cost is paid.
So in total, for a single, one-shot arrow +2 you would look at a cost of 2,500 gp and 100 days of work.
In older editions of the game powerful fiends and undead could only be hurt with weapons that had +2 or even higher enchantments, so there was a real need to obtain them. In D&D 5e even the most powerful fiends in the MM, such as a Pit Fiend, can be hurt normally by any magic weapon. There is no qualitative difference between a +1 weapon and a +2 weapon, merely a quantitative one. Hence a more economic alternative could be crafting uncommon +1 arrows instead, which cost "only" 250 gp (DMG) or 100 gp (XGE) to craft apiece.
One indeed wonders which insane wizard spent their years to create the +2 or +3 arrows that player characters might find in game.
More reasonable costs
However, here is the rule for silvered weapons, from the PHB:
You can silver a single weapon or ten pieces of Ammunition for 100 gp.
I would allow crafting 10 arrows for the cost of one item. Not only are they smaller and simpler to make than, say a sword, they also are lost as magic weapons when you use them.
Better calibrated costs
The economics for magic items are not well developed, likely because the default assumption is that they are obtained by adventuring, not bought or crafted (DMG, p.135):
Magic items are gleaned from the hoards of conquered monsters or discovered in long-lost vaults. (...)
The game assumes that the secrets of creating
the most powerful items arose centuries ago and were
then gradually lost as a result of wars, cataclysms, and
mishaps. Even uncommon items can't be easily created.
Thus, many magic items are well-preserved antiquities.
You can see this in the rarity system that is used for costs, and that does not reflect play value, even though the DMG (p. 135) claims that
Rarity provides a rough measure of an item's power relative to other magic items
For example, compare Souvereign Glue (legendary consumable, worth >25,000 gp (Magic Item Rarity, p.135 DMG), to a Bag of Holding (uncommon, worth up to 500 gp). I'd trade the glue for the bag in a jiffy, nevermind for 50 of them.
Furthermore, the cost to craft magic items is not balanced with their value in a way that would encourage crafting.
If you are open to non-canon sources and want to support crafting or magic item values that reflect item power, using Sane Magic Item Prices as the basis for the cost for various items offers a practical alternative to pricing items. Just halve these to determine the crafting cost for each. For example, the price for Ammunition +2 there is listed as 100 gp, so 1,000 gp will get you 20 arrows +2.