If everyone at the table finds it suitably cinematic, I'd say go for it. It sounds like an awesome maneuver to pull off. 4e doesn't have any general guidelines for damage from something falling on you, as shown in What happens when a creature falls on another creature?, so one would need to invent their own, possibly using the normal falling damage rules as a guideline.
That said, for the RAW answer:
(Unless you have some extremely specific circumstances.)
For a variety of (potential) reasons relating to movement restrictions.
1. Movement at the beginning of the charge
The Rules Compendium (pg 240) notes that one requirement for making a charge attack is that "Each square of movement must bring the character closer to the target," counting the distance to the target even through squares (or cubes, in this case) of blocking terrain. Unfortunately, this means one would be unable to move horizontally off the cliff edge, as by 4e's measurement system, there's no way this could be moving you closer to your target. You're also unable to move diagonally down towards them, as a character can't move diagonally through solid objects, such as the cliff you were just standing on.
Note: The remaining arguments hinge on the fall from the cliff being considered voluntary movement from purposely stepping off. If the actual falling is considered involuntary, akin to forced movement, these entire points are pretty much invalid, though point 1 still holds.
2. Total amount of movement
As you've mentioned in your question, the distance one has to fall may itself be a limiting factor. The description of Charge includes "1. Move: The creature moves up to its speed toward the target." (Emphasis added.) Presumably the hypothetical character in question doesn't have a speed of 20+, and thus would not be allowed to charge such a distance.
3. Movement at the end of the charge
Same requirement from point 1 causing a problem again, except at the other end of the charge this time. Assuming a flat surface at the bottom of the cliff you're falling off of, I can't really imagine any fall in which that would be possible, since altitude 1 and altitude 0 would both be 1 square away. If you happened to land on a ledge or slope one square higher than the target, this problem could be avoided, but that would be an extremely specific setup to have to hope for.
You might wonder if one could stop their movement at the closest possible point, altitude 1, so as to make the attack, but that would mean the charging character stopped moving in midair. Even a flight-capable creature couldn't make a charge like that with a fall, since the only way to catch yourself while falling is with an Immediate Reaction, which you can't take on your own turn and requires you to have already been flying.
The only general-purpose exception I can think of might be if you're a Pixie landing in the same space as the target, since only a Tiny character would be allowed to share a space. However, despite the Fly speed, you would still need the cliff or some other form of elevation in order to attempt your trick. Altitude limit only makes you fall at the end of your turn, and falling prone to fall requires a Minor action, so you wouldn't be able to do it in conjunction with a Standard action Charge. On the other hand, a Pixie is perfectly capable of charging downwards while flying instead of falling, if you're just looking for some Death From Above.