Start with the free Basic Rules
The basic rules are available freely online. Start with them. They are laid out extremely well, and websites are far more convenient to navigate than books. You can search, use hyperlinks, copy and paste text, easily share content with others, there are a lot of advantages.
They are also available as a pdf here.
Given you are already experienced roleplayers, the Starter set may not quite be what you want. However, it does come with an extensive adventure, pre-generated characters, some rules and dice. If you want to very quickly sit down and start playing - say within a day or two of getting it - it will let you do that.
Note that there's also the Essentials Kit, ...
The DMG is the least useful of the core books
First, welcome to 5th edition D&D, the learning curve isn't too steep for this system and you should be able to pick it up fairly easily. Before I get to the main part of the answer there is something I would like to clear up.
Unlike other systems which publisher all the important rules in a single Core ...
The Starter Set is really aimed at people who are entirely new to tabletop role playing games. Since you are not, I think you will be better off skipping it and going directly to the full game.
The full rules obviously contain more to digest, but they're also well-written and designed for people learning how to play. And, the Starter Set isn't really a ...
It depends on what you're looking for
You're comparing two entirely different products.
Starter Set: I have no idea what D&D is and want to DM
The Starter Set has everything you need to start playing. It has a bunch of pre-made characters, it has enough of the rules to get going and it has a short campaign to play through. If you simply want to get ...
Alderac Entertainment Group (AEG; Web site here and Wiki entry here) published the entirety of the Oriental Adventures line save for Oriental Adventures proper that was published by Wizards of the Coast.
2002/01 Magic of Rokugan
2002/03 Creatures of Rokugan
2002/03 Way of the Samurai
2002/05 Way of the Ninja
2002/07 Way ...