The dimensions are interior dimensions, but larger than first thoughts might indicate.
From the DMG:
This bag has an interior space considerably larger than its outside
dimensions, roughly 2 feet in diameter at the mouth and 4 feet deep.
Without a prepositional phrase after the comma, such as "which are", the dimensions should indicate the subject of the sentence: the interior space. This is strengthened by several arguments:
- Interior spaces are measured by depth, exterior dimensions by height. (A pillar is tall, but a well is deep).
- A cylinder 2 feet in diameter by 4 feet deep is less than half of the indicated diameter of the bag. (25 vs 64 cubic feet)
Further, the bag specified as being "2 feet diameter at the mouth". By qualifying that, it implies that the diameter is not uniform. Otherwise, there would be no need to say anything other than "2 feet in diameter". The bag could easily be wider inside than the opening at the top, like a goldfish bowl. This would allow for the indicated interior volume of 64 cubic feet.
Less importantly, but still something to consider, the artwork in the book fails to agree at all with the description, being more like a pouch or purse. Although this is never a final indicator, the art director would likely have rejected anything so drastically different from the description. [1]
If the interior dimension is the one indicated, the exterior could be anything. In fact, one would be hard pressed to justify how a 2 foot by 4 foot bag is intended to reduce character encumbrance. One could easily imagine the material of the bag to be stretchable to allow for a 2 foot diameter opening (it is magical and specifically created to allow for objects larger than it itself is), then any conflict with the pictured item is resolved.
Finally, since the bag is only 4 feet deep, but somehow holds 64 cubic feet, the interior dimensions could be anything that multiplies to 16 square feet: 4 x 4, 2 x 8, etc. No you couldn't likely fit a 20 foot extension ladder in the bag, but nothing in the description says that you can, either. Of course, a lenient DM could allow for anything that doesn't exceed a possible dimension. (i.e. 32 feet by .5 feet)
[1] I am a long-time illustrator in the RPG field, and I would not have been able to get away with such a disagreement, particularly with a company as large as WotC.