The asker mentions he is using a house rule:
after reading your answers and re-read the PHB. We were rolling D20 for hit on every spell then a saving throw(if applicable) on hit.
From that, it's hard to say anything about how this rule works. We have had a DM ruling something along these lines in another question. As I mention there, this is a huge nerf for spellcasters in general. About your question:
There are no (default) rules for advantage for attacking from behind 1
One option we have is an optional flanking rule, which still requires another enemy close to you in order to actually give advantage. You can read more about it here. D&D usually assumes that the character has a 360 degrees field of vision, even if they are facing a specific direction in the grid, for the purposes of being attacked. There are also optional Facing Rules (DMG p. 252), which states
A creature can normally target only creatures in its
front or side arcs. It can't see into its rear arc. This
means an attacker in the creature's rear arc makes
attack rolls against it with advantage.
It is still unclear about an attack from behind coming from a PC that is in the front arc, as it's not a situation that would be usually happening. This is not comparable to casting a spell with an actual attack roll because then the spell would be coming from the same direction as the PC (i.e., front).
So, in general...
Your DM is making a lot of house rules.
Is that a problem? Up to your group. If you, or anyone else, is feeling that these house rules are making the game less fun, unfair or similar, talk to your DM and state your concerns. From my experience, new DMs sometimes make house rules without fully understanding the mechanical consequences of these house rules. Explaining that his modified rules are hurting your fun is usually the best way to make them see it.
You also mentioned your group is new to D&D 5e, while it's not clear if that's true for the DM as well, it is possible that he is not aware that he is using house rules. If that seems a possibility for you, check with him if he knows these rules are not official. Sometimes, simply stating "Hey bro, that's not how it works" is enough for the DM to notice "Whoops, I was thinking about other random system". I myself carried misconceptions from 3.5e to 5e when it was released.
Sadly, as these are house rules, we can't answer, from the rules perspective, more than "none of these rules is stated in the books" and "your DM is the one who makes the rules, so if he said it, he's kinda right" (but he should be consistent - if your spellcasters cast a spell on the enemies' backs, you should get advantage as well). You can make a different question asking whether or not these house rules are balanced and how they impact the game overall, but, as I said, this is a different question.
As a side note, although the Shatter spell is mentioned, the actual problem seems to be about the house rule. Shatter itself could have been a fireball or any other AoE spell.
As a second side note, the Facing also states
can also change its facing as a reaction when any other
While "moves" probably means actual movement (walking from a square to another), if the DM is applying the same rule for AoE spells cast in a point of origin behind, he probably should allow you to react and change the direction you are facing to your back, when you see the spellcaster moving his hands and speaking some awkward phrases.
1 Note, however, that a specific rule is not required for Advantage/Disadvantage, as PHB p. 173 states
The DM can also decide that circumstances influence a roll in one direction or the other and grant advantage or impose disadvantage as a result.
This is more a fall-back rule and a reminder that "the DM is in right to rule anything as he wants, to be frank", though, from how I read it.