It depends on the dungeon / cave, and if your DM thinks it applies
The first Natural Explorer condition is pretty vague:
When you make an Intelligence or Wisdom check related to your favored terrain, your
proficiency bonus is doubled if you are using a skill that you’re proficient in.
The second condition is a little less vague:
While traveling for an hour or more in your favored terrain, you gain the following benefits:
So, for the first condition, what you are doing inside the terrain must be related to it. So, if you are inside of a cave/dungeon, and experience from your natural terrain directly applies to the check you are making, the double proficiency applies to it. For instance, say you are inside of a cave, and you find some minerals. Even if your DM does not classify this as in your favoured terrain, and your DM thinks that your knowledge of minerals applies, it could help you identify these minerals if the skill check is something you are proficient in. This ability is purely based on relation to your terrain, so a DM could rule that your experience in mountains qualifies for caves if they think it does, but if they don't, you don't get double proficiency.
For the second one, it states that you need to have traveled in your terrain for an hour or more. This means that the benefits apply if you have been travelling for at least an hour and if you are in of the terrain. So, if you are in a cave/dungeon in a mountainous terrain (and you have been travelling for at least an hour), these abilities apply. If the cave/dungeon is in a different terrain (as determined by your DM), they do not apply.
If a dungeon/cave is 'in the mountains', whether it is mountains at the other side of the world, or what you are familiar with, it RAW qualifies as mountainous terrain, so it qualifies. If it is 'a vast subterranean realm inhabited by drow, mindflayers, aboleth, and other strange, sinister creatures' (definition from Wikipedia), it is underdark, etc.