First, as Miniman points out, the evil Mage shouldn't be allowed to maintain concentration on both Fly and Greater Invisibility at the same time.
Second, Jeremy Crawford has ruled (via Twitter) that while you detect the presence of a magically invisible creature within 30' you can't see its aura unless you can see the creature.
Detect magic lets you see an aura only around a visible creature or object, not around an illusion. But you can sense the magic.
So if the Mage had only cast Greater Invisibility, a PC which casts Detect Magic would know that there is some form of magic within 30' but not see an aura for the Mage (or his wand). The sensation of magic could be caused by anything though, including PC magic items or ongoing spells, so this may not be helpful. The PC would not know which square the Mage was in from Detect Magic since he wouldn't see an aura.
Third, the Mage is not guaranteed to be silent via the Fly spell or the Invisibility spell.
However, the Mage should roll a Stealth check (hide as an Action) if he doesn't want the PCs to know where he is. Every time he attacks or makes significant noise (such as spellcasting), he should have to spend a new Action to hide. If a PC's passive perception beats/equals the Mage's Stealth check, then the PC knows what square the Mage is in and can use an attack or spell which does not require sight (many non-AoEs spells with saves require that the target be seen). Attacks (and spells with attacks) will have disadvantage but spells with saving throws will not. A PC can also spend an action to roll an active perception check to try to beat the Mage's Stealth (if their passive wasn't good enough).
Every time the Mage takes any damage they will need to make a Concentration check to remain invisible. The PCs will have to rely on a combination of multiple hits to force minimum Concentration saves and possibly a few big damage (>= 22 damage) hits to force a Concentration save with a higher DC.
A PC which knows which square the Mage is in can:
- Attack with a weapon at disadvantage. (Each attack that hits will cause a separate Concentration check.)
- Cast an attack spell such as Eldritch Blast or Shocking Grasp or Scorching Ray with Disadvantage. (Each attack that hits will cause a separate Concentration check.)
- Attack with Spiritual Weapon at disadvantage (it targets "a creature within 5 feet of the weapon").
- Cast Fireball at the Mage's square. The save is rolled normally.
- Cast Acid Splash normally (it targets "one creature within range").
A PC cannot:
- Cast a spell like Poison Spray or Sacred Flame at the Mage since it targets "a creature you can see."
A PC can probably figure out which square to target by watching where their allies are targeting or being told even if they couldn't figure it out.