It seems like this question boils down to determining what constitutes casting a spell or making an attack.
Casting a spell (PHB, p. 202):
Certain spells (including spells cast as rituals) require more time to cast: minutes or even hours. When you cast a spell with a casting time longer than a single action or reaction, you must spend your action each turn casting the spell, and you must maintain your concentration while you do so (see “Concentration” below).
Note the emphasized text: as soon as you start taking the Cast a Spell action, you are casting a spell. If that were not the case (for example, if you were only "casting" find familiar at the end of the hour), then the rules would read something like "In order to cast a spell with a casting time longer than a singe action ...".
Also note the concentration requirement — invisibility requires concentration, so even without the above interpreation, you would need to drop concentration to cast a multi-action spell like find familiar or find steed. This is also true of greater invisibility, which doesn't otherwise end when you cast a spell.
As for an invisible character casting a single-action spell while another character has taken the Ready action to (for example) fire an arrow at them once they become visible: this seems to fall under the rules for adjudicating the timing of reactions (DMG, p. 252):
If a reaction has no timing specified, or the timing is unclear, the reaction occurs after its trigger finishes, as in the Ready action.
So any reaction to an invisible spellcaster becoming visible as a result of casting a spell would occur after the spell takes effect.
Making an attack
We have the general rule about what constitutes an attack (PHB p.193):
If there’s ever any question whether something you’re doing counts as an attack, the rule is simple: if you’re making an attack roll, you’re making an attack.
For the purpose of this discussion, we can re-formulate that as when you make an attack roll, you're making an attack. The most significant game-mechanical implication of this with regard to invisibility is that an invisible attacker becomes visible even if the attack misses.
Preparing an attack, however that may be represented in game mechanics (loading a crossbow bolt, taking the Ready action, etc.), isn't making an attack roll, and thus wouldn't cause invisibility to end.
However, considering the actual actions of the characters, it's not well-defined when the attack roll happens in the course of combat, or exactly what it represents. A pair of fighters dueling with swords will, in the course of a six-second combat round, exchange several plays, remedies, counter-remedies, and other motions; they may represented by one or more "attacks" in the game-mechanical sense, but there will be many more weapon swings involved than attacks rolled. So the motion of a weapon (or limb) towards an opponent doesn't necessarily coincide with the exact moment that the die-rolling part of the attack happens.
Nevertheless, the distinction seems largely cosmetic for most purposes. Consider what's being narrated to a sword-wielding hero being struck by an invisible assassin.
If invisibility drops only as the damaging part of the attack starts:
You feel a sudden blow to your sword, swiping it aside, and then see a cloaked figure appear before you, stabbing towards your exposed chest with a shortsword.
Or if it drops just before any hostile action starts:
A cloaked figure appears before you, wielding a shortsword; they swat your blade aside before you can react and step inside your guard, stabbing towards your exposed chest.
The difference is an interesting narrative detail, but not one that's relevant for any game-mechanical interactions.