I would like to clarify another portion of the answer to your question, and make sure you understand how your proposed course of action would occur.
-First, You use a minor (potion), becoming invisible
-Second, you use a stealth check at the end of a move action to successfully avoid being percieved.
-Third, you use a standard action to ready your action, with the following trigger "I will attack with my readied action as soon as the caster casts a spell." Functionally this is clear, but mechanically, let's say you are referring to any power, utility, attack, or otherwise non-move action. Though I believe you are referring to an attack power."
You have successfully used up all actions for your turn, and have successfully readied a valid attack action. You then wait if the trigger occurs.
If the trigger does not occur before your next turn, your readied action is lost, and you must ready again as before in order to do this again.
If the trigger DOES occur before your next turn, because your readied action is "when the target attacks," your attack will actually happen AFTER the target caster's attack occurs! This is explicitly in the most recent Compendium (DnD Insider) rules as of 5/22/12. The rules say that the act of taking a readied action on another players turn is an IMMEDIATE REACTION to the trigger, and will occur in response to the trigger, not before it. Of note, there is a specific exception to this only in the case of a move action. You can interrupt a characters move action to take your readied action, if your trigger you set is to the target's move action, the trigger occurs, and only after the target has at least moved one square of the triggered move action. The target's move is interrupted, your readied action occurs, and then the creature resumes its move assuming it is still able to do so.
--> In addition, your new initiative would reset to before the caster's turn, and this can leave you without a full set of turns for longer than if you had not taken the readied action, especially if the caster's turn is normally immediately before you in the initiative!
In the case of the caster and you above, I can't think of any trigger that you could set that would reliably allow you to attack before the caster took its attack action, because you don't know when it's going to attack, and once you know, you can do nothing to prevent it because your readied action occurs after the triggering attack. The only reliable way to do damage to the caster BEFORE he attacks is to attack him during YOUR turn, instead of readying the action.
An example of taking a readied action in relation to a move, for clarity:
-You are almost dead, and in a narrow, two square-wide alley with allies behind you and enemies in front. Let's say you would like to stay in front, but you want to move away in response to an enemy approaching you (say to make an attack), in order to avoid being attacked. You could ready a move action, saying you will move away (behind your allies), in response to any enemy's move that brings that enemy to within two squares of you. Note that you must use a STANDARD action to ready your move action, per the rules. Now, The next time an enemy approaches within 2 squares of you on your turn, you can retreat to behind your allies, safely avoiding a melee attack.
--> This example is not foolproof to prevent an attack from damaging you. Unfortunately, if the DM is mean, the enemies can still hit you from any number of means: a ranged attack, a melee attack with melee range 3 (which is very rare!), an area attack, or a close blast / burst attack with range of 3 or more.