You must prevent a Hag from taking an action, or remove the Heartstone from their possession, in order to stop them from becoming Ethereal.
Spellcasting vs Actions
The Hag has an ability to take the action "Etherealness" as quoted in your question. The description does not state that this is spellcasting, but only requires their ability to take an action and to be in possession of the heartstone.
Had this been spellcasting, it would have been listed under at-will or innate spellcasting within their stat block (either the Hag's or if using the Variant rule for Hag Covens) and would require somatic and verbal components (see below.)
Holding vs Having
As long as the Heartstone is possessed (not held, just has) and the Hag can still take an action (doesn't have the condition Incapacitated), they can become Ethereal.
This is similar to the idea that you can have a shield in your possession, but that doesn't give you the benefits of the shield. In this case, possession is all that's necessary. Had the Monster Manual said "hold" or "wield", it would have required that.
Innate spellcasting and components
Whether or not components are required for a creature's innate spellcasting will be determined by their stat block. A Hag does not require material components to cast their spells, but do require somatic and verbal. A bound and gagged Hag would be unable to cast their spells that require those components.
If you are thinking about whether or not a bound/gagged creature has been Incapacited, consider that conditions like Grappled and Restrained do NOT remove the ability for a creature to take an Action. Being Stunned, Paralyzed, or Unconscious are the types of conditions that prevent Actions.
Preventing Etherealness as an Action
The best way to do this in the case of the hag is to remove that Heartstone once the creature has been bound/gagged (or better...before!) Otherwise, not only do you need to restrain and gag them, but you will also need to make sure they are either Stunned, Paralyzed, or Unconscious by either magic or reducing their HP to 0.