All Fate Points spent to propose or refuse a compel are lost.
Just spend them and they're gone. The GM doesn't pay to compel, ever, and players don't pay to compel their own character. If the GM pays to refuse a compel on one of their characters, it comes out of the scene pool.
Informally speaking, if you think a compel would apply to someone else's character and it doesn't work to your advantage at all, you can just ask them to propose it on themselves and they won't pay anything to do it.
Fate Points awarded for accepting a compel come from the GM's infinite reserve and are usable immediately.
Accepting a compel is not, strictly speaking, a transfer of Fate Points from one place to another, even though that's the practical result of it most times. Thinking of them all as coming out of the GM's infinite reserve makes it clearer when they're coming from when the compel was unpaid in the first place.
If a compel's not judged valid, there's nothing to propose and nothing to pay.
To illustrate the difference between derailing a compel at step 2 and rejecting a compel at step 3, consider the following:
Starhound, Athens, and Twilliam fight their way to the core of Dark Stobolous's planet-cracker and they're all pretty badly banged up. Dark Stobolous hasn't shown his shapeless black helmet yet and the drama compass is pointing due S(howdown).
Twilliam's player: "We found out Dark Stobolous was Powered By a Black Hole, right? Well, uh-" :waving Fate Point: "-suppose the containment on that thing is failing and he's got to get tended to by maintenance so he can't be here to fight us, darn the luck?"
GM: "No, that's not really something that happens to Dark Stobolous. Nice try, though."
The GM does not take the Fate Point from Twilliam's player. Starhound's player then waves a Fate Point.
Starhound's player: "But the planet-cracker's A Singular Engine Of Destruction. And you love puns way too much. Is it even safe for two black holes to be that close to each other?"
GM: "Right now, sure. Why don't you just hold onto that until after you cause a little ruckus?"
The GM doesn't take Starhound's Fate Point, either.
Dark Stobolous shows up and a Conflict starts, the PCs throwing everything at both him and the planet cracker. Eventually...
GM: "Oof, that's definitely a consequence. I think you've managed some Cracked Nictoplate Shielding..."
Athens's player: :waving a Fate Point: "That sounds like it'd be a real problem for someone who's Powered By a Black Hole?"
GM: "Hm! So it would. But you know what? I think this is too important to Dark Stobolous for that to matter, right now. ...dangit, I had plans for that."
The GM takes Athens' Fate Point and the last point from the scene pool, and drops them back into the central pile that serves as the GM reserve.
Okay, so, what's the difference? Don't both sides pay a Fate Point every time someone rejects a compel? Well, yes. But that only happened once. Only Athens's player actually proposed a compel. The other two made what's best understood as a mistake in play.
Suppose for a second Twilliam's player couldn't read the GM's handwriting, and waved around a Fate Point suggesting that Dark Stobolous wouldn't show up in the core of the planet-cracker because he was busy grooming the Black Mole that was the source of his powers. That couldn't possibly happen, because Dark Stobolous isn't powered by a Black Mole, and so Twilliam's player wasn't actually proposing a compel - a specific, plausible course of action suggested by an aspect.
In much the same way, the first two attempts - Twilliam's player's half-joking attempt and Starhound's player's better-grounded one - were not plausible courses of action, in the GM's estimation of how Dark Stobolous's Aspect was supposed to function. Neither one was actually a compel in the first place, so it didn't cost a Fate Point to propose or to reject them.
But if I don't pay to try an invalid compel, can't I just free-associate for two hours and waste everyone's time?
Sure. And the GM never pays, so even if they "had to pay" they could just free-associate for two hours trying to get Starhound arrested because Two Blasters, No Waiting obviously means he shot up a Spac-E-Mart when he wanted a Nebula Freeze but the line at the register was too long.
But the rules of Fate are assuming that everyone at the table is playing in good faith, so the GM isn't going to do that and neither are you. If you try a compel on something that isn't really part of an Aspect the way its creator envisioned it, well, you made an honest mistake, and why should you be punished? Otherwise people would be incentivized to disagree with you because it would just waste your Fate Points for nothing.