How we handled it in our first campaign.
We never did get rid of that !@%#*! Night hag, so every night a couple of us (to include my Cleric) slept in the same room or place as the Bard (that is who the Hag had a grudge against) and when the hauntings began (evidenced by the Bard's reaction to the night mare) my cleric would cast protection from evil. I had to be sure to always save a 1st level spell slot for this. (What this basically did for our party is reduce how many first level slots I could cast by 1 on some adventure days, depending upon when the haunting happened over the night, early watch or late watch).
That was how our DM ruled it: contact initiated, respond with protection from evil spell, contact broken. I did not find any conflict with the book when I dug into it later.
I twice tried to trap her with a Magic Circle, but in neither case did it work, I think due to the Hag figuring out what I was doing and staying on the Ethereal Plane. So we stopped trying that. I was not on the same plane of existence as the Hag.
We were not high enough level for anyone to turn Ethereal and attack her, so the Hag was a great frustration to the party. When the game went dormant (around level 6) we still had not solved the Hag problem, and had burned some loot/gold on a greater restoration spell for the Bard to get him back up to scratch.
Why this ruling makes sense
Had the DM ruled that the protection from evil spell would not disrupt the contact, this haunting would have been a case of an "easy button" killing of the bard with nothing a level 5 party could do to stop it.
The Paladin(Vengeance), the Wizard(Transutation) and I (Tempest Cleric) were discussing a variety of ways to try and contact the Hag to make a deal to get her to stop bothering the Bard, or to try and trap her, but we ended up in some other adventures before we could get that plan to crystalize. Then the game went dormant due to RL.
Leomund's Tiny Hut (ritual)
If you have an arcane caster who has Leomund's Tiny Hut, it can be cast as a ritual and prevent access by the Hag.
I'd need to go back to game logs for this, but I think once or twice we slept in the Bard's Leomund's Tiny Hut and didn't get the haunting. Having to save a 3rd level spell for that every day/night (for a level 5 and then level 6 party) was a larger resource debt than we wanted to pay, though it was effective). At the time, the ability to cast LTH (ritual) by the Bard (and not burn a slot) went un-noticed. (Thanks for making that point, @markovchain).