I don't recall seeing anything about caltrops obtained from a Bag of Endless Caltrops disappearing. Does that mean you could in theory stockpile these caltrops by regularly removing them from the bag, storing them separately, letting the bag's supply regenerate, and repeating?
Yes, you can.
As you say, the caltrops are not described as ever disappearing, and are in fact stated to “follow all the rules of for normal caltrops.” The item requires Leomund’s secret chest to make, which implies that it’s basically just literally an extradimensional space full of perfectly normal, mundane caltrops that you can draw at will. Maybe there’s a Hemi-demi-semi-plane of Caltrops somewhere.
At any rate, it’s not much of an exploit; caltrops aren’t exactly amazing. The sheer amount of time this takes prevents you from doing much with it most situations.
I don't see why not. Generally nonmagical objects created by conjuration spells are permanent. The bag radiates conjuration magic; the caltrops produced are nonmagical.
The bag costs 800gp, 5 handfuls of caltrops would cost 5gp. No balance concern there. (And if you want to consider the economic ramifications of magic in the D&D verse, this bag is the least of your problems.)