If I put something in a mundane bag and then carry the bag, I am clearly carrying the things in the bag. No reasonable person would argue that I'm not carrying the items in my backpack on the basis that my backpack is carrying the items and I'm carrying my backpack.
There are no rules that specifically say that things in a Bag of Holding aren't being carried by the person carrying the Bag. In the absence of a specific rule to the contrary, we should probably assume the Bag of Holding counts as a normal bag except for the explicit mechanical effects of being bigger on the inside and having a constant weight.
Still, it seems entirely reasonable that an item that's in an extradimensional space -- functionally another plane of existence -- isn't really on your person in any meaningful sense. If items in the bag are on a whole different plane from your body, how can you be said to be carrying them? (Technically, the Bag of Holding itself doesn't claim to be an extradimensional space; that's an inference based on similar items that do, and stated directly elsewhere in the rule books, but not in the item description. However, it's generally accepted that this is so.)
But going the other way, if an item is within arm's reach in a bag on your back, how can it not be said to be on your person?
Ultimately there is not a clear answer to this. Your DM will have to make that decision, and it's really not possible to predict what any given DM will decide. So if you really want to make sure an item doesn't get magically yoinked (that's a technical term) when you aren't looking, maybe best to keep it in a mundane container.