It takes some assumptions to make a ruling here.
First, what does the curse do?
Curse. This axe is cursed, and becoming attuned to it extends the curse to you. As long as you remain cursed, you are unwilling to part with the axe, keeping it within reach at all times. You also have disadvantage on attack rolls with weapons other than this one, unless no foe is within 60 feet of you that you can see or hear.
This seems fairly straightforward, but the next paragraphs seems to be a follow-on as much as a new feature.
Whenever a hostile creature damages you while the axe is in your possession, you must succeed on a DC 15 Wisdom saving throw or go berserk...
Certainly this seems to be the bulk of the curse. I would rule that this really is the curse, and that the stuff in the 1st curse paragraph is just a detrimental feature. I could understand a ruling that took this paragraph as a separate feature not connected to attunement or the curse in general, but that doesn't feel right. It's the berserking thing that is the curse (it's in the name, right?).
Second, what can we do with the Axe if not attuned?
Other answers posit that because the general rule for magic items state the if not attuned, a character gains none of the magical benefits.
Some magic items require a creature to form a bond with them before their magical Properties can be used. This bond is called attunement [...] Without becoming attuned to an item that requires attunement, a creature gains only its nonmagical benefits, unless its description states otherwise.
I argue, however, that the description over-rides this general rule:
You gain a +1 bonus to attack and damage rolls made with this magic weapon. In addition, while you are attuned to this weapon, your hit point maximum increases by 1 for each level you have attained.
The description specifically calls out extra features that happen if you are attuned. This infers that if you are not attuned, it functions as a +1 weapon. It's not the clearest explanation, but my ruling would be that this is a case of the specific over-riding the general.
So, all that said, based on my above rulings and assumptions,
If you are not attuned, you have a +1 Axe, and getting hit while holding it does nothing.