By default yes, all of the worlds in 5e exist in the same multiverse
The spell Dream of the Blue Veil from Tasha's Cauldron of Everything explicitly sets all worlds as being in the same material plane:
You and up to eight willing creatures within range fall unconscious for the spell’s duration and experience visions of another world on the Material Plane, such as Oerth, Toril, Krynn, or Eberron. [...]
Beyond this spell, we have other evidence for this being the case. D&D 5e is set in a multiverse by default, which contains many worlds. This is explicitly called out in the Gods of the Multiverse appendix of the Players Handbook (PHB):
Religion is an important part of life in the worlds of the D&D multiverse.
[...]
Many people in the worlds of D&D worship different gods at different times and circumstances.
Other worlds are explicitly called out in the D&D Pantheons section of this appendix:
Each world in the D&D multiverse has its own pantheons of deities, ranging in size from the teeming pantheons of the Forgotten Realms and Greyhawk to the more focused religions of Eberron and Dragonlance. [...]
The Dungeon Masters Guide (DMG) also explicitly calls out that there are multiple worlds in the Material Plane in the Making a Multiverse chapter:
[...] In this context, the Material Plane is the nexus where all these philosophical and elemental forces collide in the jumbled existence of mortal life and matter. The worlds of D&D exist within the Material Plane, making it the starting point for most campaigns and adventures. The rest of the multiverse is defined in relation to the Material Plane. [...]
When talking about the Material Plane, this chapter also has a section called Known Worlds of the Material Plane which states (along with more in depth per world descriptions):
Worlds of the Material Plane are infinitely diverse. The most widely known worlds are the ones that have been published as official campaign settings for the D&D game over the years. If your campaign takes place on one of these worlds, that world belongs to you in your campaign. Your version of the world can diverge wildly from what’s in print.
Chapter 11 of the DMG A World of your Own it reinforces this idea:
This book, the Player’s Handbook, and the Monster Manual present the default assumptions for how the worlds of D&D work. Among the established settings of D&D, the Forgotten Realms, Greyhawk, Dragonlance, and Mystara don’t stray very far from those assumptions. Settings such as Dark Sun, Eberron, Ravenloft, Spelljammer, and Planescape venture further away from that baseline. As you create your own world, it’s up to you to decide where on the spectrum you want your world to fall.
If you needed more evidence of this, the adventure Dungeon of the Mad Mage (set on Faerun) makes named worlds of other campaign settings explicitly set within the same multiverse:
In room 15b on Level 9 of the Dungeon, the following treasure is found (emphasis mine):
The arcanaloth is an avid reader and has collected countless books from across the multiverse. Most of the books cover mundane subjects such as etiquette, oratory, and poetry. Twenty of the books are treatises on the Outer Planes and chronicles of historical events on various Material Plane worlds, including Toril, Oerth, Athas, and others; these tomes are worth 100 gp each to an interested buyer. A character who spends 1 hour searching can find one of these rare tomes.