The wording on the Etherealness spell is contradictory
From the Etherealness spell:
... You remain in the Border Ethereal for the duration or until you use your action to dismiss the spell ...
As you point out, the description in the PHB states:
Etherealness allows adventurers to enter the Ethereal Plane and travel from there to any of the planes it touches—the Shadowfell, the Feywild, or the Elemental Planes.
The description of the spell seems to suggest that you must remain in the Border Ethereal for the duration of the spell, whereas the wording in the PHB suggests that once you are in the Border Ethereal you are free to explore into other planes as you wish.
There is probably a reasonable middle ground
I'd probably rule it so that you can use Etherealness to enter the Border Ethereal, and then travel to the Feywild or the Shadowfell. At the end of the duration of the spell, you could return to that corresponding location on the Material Plane.
It becomes trickier if you enter the Elemental Planes or any of the Outer Planes during your time in the Border Ethereal. There is no mapping of locations between the Outer Planes and the Elemental Planes as there is with the Feywild and the Shadowfell. So it would be tricky to determine how the end of the spell would work in these situations. You could rule it so that the Etherealness spell would break, trapping a character in one of these other planes.
There is a second part to your question though, can the Ethereal Plane contain portals to other planes?
From the DMG on Planar Travel (emphasis mine):
“Portal” is a general term for a stationary interplanar connection that links a specific location on one plane to a specific location on another.
And from a little further on:
Most portals exist in distant locations, and a portal’s location often has thematic similarities to the plane it leads to. For example, a portal to the heavenly mountain of Celestia might be located on a mountain peak.
This shows that a portal to the Outer Planes can exist on the Material Plane. There aren't any specific limitations for which plane can have which portals. The Ethereal Plane can have portals to any other plane of existence.
In the scenario above, there could simply be a portal to Celestia at the top of a mountain both in the Ethereal Plane and on the Material Plane
Specifically, there are Ethereal Curtains in the Deep Ethereal that act as portals to other Inner Planes.
From the DMG on the Ethereal Plane (emphasis mine):
...Scattered throughout the plane are curtains of vaporous color, and passing through a curtain leads a traveler to a region of the Border Ethereal connected to a specific Inner Plane, the Material Plane, the Feywild, or the Shadowfell. The color of the curtain indicates the plane whose Border Ethereal the curtain conceals...
A caster of Etherealness does not enter the Deep Ethereal, so they would not see the curtains. If you cast Plane Shift or Gate (or access the Deep Ethereal in another way) you can use these curtains to travel to the Border Ethereal of the Shadowfell, Feywild, or the Elemental Planes.
