Canonically, yes, but a DM may decide otherwise.
The description of Shadow Crossings in the DMG actually has the answer:
Similar to fey crossings, shadow crossings are locations where the veil between the Material Plane and the Shadowfell is so thin that creatures can walk from one plane to the other.
Shadow Crossings allow creatures to walk from one plane to the other, but note the first phrase: similar to fey crossings. This property is what makes the two types of crossings similar.
In Ghosts of Saltmarsh, there is a hull upgrade available to ships called Living Vessel:
This vessel’s hull was crafted in the Feywild under the direction of master eladrin shipwrights. It is a living plant, drawing sustenance from water and sunlight. Vines covered in thick leaves hang over its side, and the wood runs with fresh sap when damaged. The ship gains a +2 bonus to all Constitution checks or saving throws. As long as the ship has at least 1 hit point, it regains 10 hit points every minute.
The ship itself is a plant that is native to the Feywild, so there must be some means of access to materials in the Feywild for this upgrade to be available at all.
Additionally, 4th Edition has an entire book dedicated to the Feywild: Player's Option: Heroes of the Feywild.
In the description of the Satyr playable race, we see some things that guide us to an answer. On page 34,
Possessed of mischievous and investigative souls,
satyrs ventured cautiously through the fey crossings
and made contact with the fledgling races of the
[...] Their race has become well known to the barbarian
tribes and civilized outposts along the borderlands,
where they emerge from fey crossings to sate their
curiosity regarding the mortal realm.
This demonstrates that at least some Fey crossings can take you from the Feywild to the Material plane.
From the many descriptions of Fey crossings found in Heroes of the Feywild, the intent certainly seems to be that they are two-way crossings. They seem to be places where the Fey influence our world, and when the crossing is open, we can travel to their's and they can come explore ours. In the introduction (pg. 9), we see this description of Fey Crossing Hamlet's, small towns situated in close proximity to a crossing:
But as long as you have lived here, you have
noticed something strange and magical about the
hamlet. When pestilence and famine strike other
nearby settlements, your crops grow tall and strong,
wild game is plentiful, and your water remains pure
and clean. The mists that rise in the evening suggest
shapes of fantastic beasts such as unicorns and griffons. Sometimes after dusk in the moonlit woods and
fields beyond your village, you hear strange sounds
like the tinkling laughter of faraway bells, and you
see distant colorful lights dancing slowly through the
air like fireflies. Sometimes strangers happen by your
hamlet, smelling of faraway scents that create cloud
castles in your mind; they speak of domains of eternal
summer that are home to elf maidens of surpassing
Either way, a Fey crossing is a world building tool like any other means of travel. The DM, in building their world decide what methods of travel connect two places and how, so the DM may decide a particular Fey crossing is a one-way ticket.