You've thought about this. You've played in a campaign allowing it (albeit the effects were capriciously adjudicated). Thus if you want to try this, despite KRyan's accurate and firm warning against it, you should. Here's one way to do so.
With caution and careful management, metamagic changing a spell's casting time from a 1-round action or less to an immediate action could adjust a spell's level by +8 (but see below)
Casters are already the most powerful beings in the game. A feat allowing high-level casters to cast low-level spells off-turn probably won't significantly change the game's balance. But making such an ability easy and obvious is hazardous because then the ability can be exploited easily and obviously. That makes unbalanced a lone feat changing a spell's casting time from a standard action to an immediate action, even if the feat increases the spell's slot by +8. Such a feat would, for example, lend itself to extrapolated metamagic rods and metamagic-level-reducing shenanigans.
But a feat that changes a spell's casting time from 1 swift action to 1 immediate action, on its surface, isn't particularly unbalanced, especially if limited to high-level casters. For example, consider this homebrew feat:
You can cast spells even faster.
Prerequisites: Spellcraft 15 ranks, Quicken Spell.
Benefit: A spell modified by this metamagic feat can be cast as an immediate action. Only spells with casting times of 1 swift action can be modified by this feat. Casting a spell modified by this feat doesn't provoke attacks of opportunity.
Level Adjustment: +4.
Special: You can apply the effects of this feat to a spell cast spontaneously without increasing the spell's casting time.
(Note that while I have not playtested this feat, I've played a fair amount of high-level Dungeons and Dragons 3.5 and Pathfinder campaigns. My personal experiences in such campaigns color the remainder of this answer. Also, such a feat should not exist in a Dungeons and Dragons 3.5 campaign.)
On its face, the feat is limited to...
- 0-level through 5th-level spells with casting times of 1 swift action,1 and...
- 0-level and 1st-level spells with casting times of 1 round or less that are already modified by the feat Quicken Spell.
For most campaigns, such effects shouldn't be a problem. Most campaigns never even see level 15 characters (or end soon after seeing them), and a Wiz17 who uses his highest-level spell slot to cast an off-turn grease or color spray just isn't disrupting the game with his abilities. However, not all campaigns are most campaigns.
Pathfinder contains the bloodrager archetype metamagic rager, the oracle archetype seeker's supernatural ability seeker magic,2 the feat Spell Perfection,3,4 the trait Magical Lineage,3 the trait Metamagic Master3 (né Waywang Spellhunter), the spell arcane concordance,5 and, doubtlessly, will contain other effects in future supplements.
The presence of these effects means such a homebrew feat must be carefully monitored by the GM, requiring the GM to consider carefully if, when these already present and future effects combine with his homebrew feat, his homebrew feat grants to casters power far beyond that already possessed.
With such careful management and attention the feat's just another possible interesting tool in the caster's toolbox rather than the only or best tool in the caster's toolbox.
1 I checked spells with Casting Time: 1 swift action and while some are kind of spiffy off-turn (e.g. the 2nd-level Clr spell peacemaker's parley, the 6th-level Sor/Wiz spell cold ice strike, the 9th-level Sor/Wiz spell ride the lightning, the litany of line of spells) none appear horribly unbalanced as immediate action spells. (This could, of course, change with the next Pathfinder supplement.)
2 At level 15 this archetype allows the casting of 0-level through 6th-level spells and 0-level through 4th-level spells, respectively, when such spells are granted by the oracle's mystery. The reduction doesn't combine with other reductions, however.
3 When picked affects 1 named spell ever. Choose wisely.
4 The feat Spell Perfection has as a prerequisite 15 ranks in the skill Spellcraft (and 3 metamagic feats), likely limiting it to characters of level 15 or higher.
5 Limited to small set of metamagic feats; this spell is used here as a placeholder for other effects employing similar mechanics (e.g. the amulet of grasping souls, the incense of meditation).