Before (unfortunately). The trigger is
[being] subjected to an effect that allows you to make a Dexterity saving throw to take only half damage
And that is when the decision must be made: when the effect is hitting you. If the decision time was after rolling the save, the trigger would be "after making the roll" or similar.
But it says "if you succeed"! Doesn't that means I can choose to use it only if the roll succeeds?
There are two ifs in the description. The first is part of the trigger, the second is part of the effect.
Trigger:
If you are subjected to an effect that allows you to make a Dexterity saving throw to take only half damage…
Choice, after the trigger:
… you can use your reaction to…
Effect, after you've decided to take advantage of it:
… take no damage if you succeed on the saving throw, interposing your shield between yourself and the source of the effect.
So the order of operations is
You are the subject of an effect that allows a Dex save for half damage and the ability triggers
You get to choose whether to use Shield Master's optional effect
You roll the Dex save and apply the effect:
- If you make the save, you take no damage
- Otherwise you take full damage
But doesn't that make it awful?
No more or less than an opportunity attack. When you get a chance to attack with a reaction, you're not guaranteed success, you're just getting an opportunity to try something. An opportunity attack uses up your reaction before the roll, but that chance of it having no benefit isn't controversial.
This use of your reaction is the same: using up your reaction creates an opportunity to get a benefit, but whether the spent reaction actually gets you a benefit depends on whether your roll afterwards succeeds.
And since it's only ⅓ of Shield Master's effects, it doesn't have to be more amazing than a regular reaction in order for the feat to measure up against the other feats — and it is a kind of reaction that is otherwise inaccessible, which itself has special value. It is not an amazing benefit, but it's not awful either, especially since it statistically improves your durability against such effects regardless of any one roll being a save or fail.