I have a guest player who is secretly an evil "plant" inside the party causing trouble. He's a sorcerer who wants to use Subtle Spell metamagic to stop the party from delving into his master's old lair, and he has a custom ring item that works as a hands-free arcane focus. He and I agreed that he'd give me signals about what to cast and where, and I'd roll the damage and keep track of the lost sorcerer points.
So the party is sneaking past a sleeping giant and the sorcerer signals fireball on the party to wake the giant. After the battle, the cleric says her passive Perception (which is pretty high, admittedly) should have let her see where the fireball originated from, pointing out that fireball says:
A bright streak flashes from your pointing finger to a point you choose within range
Now, I assume the pointing finger is part of the somatic components of fireball and thus not necessary with the Subtle Spell metamagic. And for spells that obviously originate from the caster like cone attacks, it wouldn't be particularly possible to hide the location of the caster. But fireball's streak effect doesn't seem to be a mechanic of the spell, more just flavor text.
I addressed the problem in-game by rolling a Sleight of Hand check against the cleric's passive Perception using the sorcerer's stats and managed to keep it hidden and "it happened so fast you saw it came from behind you but nothing else", but is that the right solution?
But the more I think about it, the more complicated it seems. Does it follow the stealth rules where any attack, even long distance arrows, immediately gives away the position of the attacker? Would a subtle psychic scream (XGtE, p. 163) be traceable to the caster despite having no visible ties to the caster?