This probably doesn't work…
The 2nd-level Sor/Wiz spell rope trick [conj] (Player's Handbook 273) likely can't be modified by the metamagic feat Reach Spell (Complete Divine 84) because the spell rope trick doesn't have the standard range entry Range: Touch entry that's needed for it to be considered an actual touch spell. That is, instead of the entry Range: Touch, the spell rope trick has the nonstandard range entry Target: One touched piece of rope from 5 ft. to 30 ft. long.
So it's likely that the 2nd-level Sor/Wiz spell rope trick [conj] (Player's Handbook 273) that's modified by the metamagic feats Reach Spell (Complete Divine 84) and Transdimensional Spell (Complete Arcane 84) (hence typically making the spell occupy or expend a 5th-level spell slot) just fails even if a caster outside the rope trick effect uses an effect to see inside the rope trick effect to aim the spell at a loose rope that a foolish creature therein left lying on the extradimensional space's virtual floor.
(This DM would've told the caster who was trying to prepare the modified spell that the Reach Spell feat's benefit couldn't be applied to the rope trick spell, and would've told a caster that casts his spells without preparation that he knew instinctively that the spell couldn't be modified with that feat's benefit. Likewise, this DM would allow any creature that was already familiar with the spell to make a Knowledge (arcana) skill check (DC 5 and maybe even 0) to realize the same thing.)
…And if it does, the DM says what—if anything—occurs
If the DM can be persuaded that a caster can apply the benefit of the metamagic feat Reach Spell to the spell rope trick despite the spell's nonstandard Range entry—maybe you brought beer—, and everything else is going that caster's way, that caster can cause
one end of the rope [to] rise… into the air until the whole rope hangs perpendicular to the ground, as if affixed at the upper end. The upper end is, in fact, fastened to an extradimensional space that is outside the multiverse of extradimensional spaces…. (PH 273)
This would, of course, create in the extradimensional space an extradimensional space, which the rope trick spell warns readers is bad: "It is hazardous to create an extradimensional space within an existing extradimensional space or to take an extradimensional space into an existing one" (ibid.) Exactly how bad is the DM's call: the game never follows up this pregnant warning with rules except the specific case of the interaction between a bag of holding and a portable hole.
In fact, in the Rules of the Game Web column "Carrying Things (Part Three)" one of the game's co-designers, Skip Williams, says it's okay to disregard the dire warning accompanying the spell rope trick:
It's best to treat a Heward's handy haversack as a bag of holding when it interacts with a portable hole.
Other interactions between extradimensional spaces are possible. For example, the rope trick and Mordenkainen's magnificent mansion both create extradimensional spaces. The rope trick spell description makes a passing mention of "hazards" associated with placing one extradimensional space inside another, but gives no details.…
I recommend that you ignore this reference. Your campaign won't be improved if rope trick effects implode when someone carries a bag of holding or portable hole inside. A Mordenkainen's magnificent mansion should likewise prove benign if someone carries a bag of holding or portable hole inside.
So, in the same way somehow tossing a bag of holding into a rope trick effect probably won't explode everybody hiding therein, zapping a loose rope in rope trick effect with a reach transdimensional rope trick spell likely shouldn't, for example, create a rift to the Astral Plane and cause the creatures within the rope trick effect to be forever lost.
Nonetheless, ask the DM. The DM may ignore the Rules of the Game columns (the columns do have a sort of infamous reputation despite sometimes offering otherwise unavailable clarifications), and then he can totally rule differently and make something spectacular happen… especially if you brought a lot of beer.