TLDR: While levitating, a creature is not falling, therefore the rules for falling damage cannot be applied.
To answer your question, lets look at the rules for the two things involved here: Levitate and falling
Levitate (PHB 255)
One creature or object of your choice that you can see within range
rises vertically, up to 20 feet, and remains suspended there for the
duration. The spell can levitate a target that weighs up to 500
pounds. An unwilling creature that succeeds on a Constitution saving
throw is unaffected.
The target can move only by pushing or pulling
against a fixed object or surface within reach (such as a wall or a
ceiling), which allows it to move as if it were climbing. You can
change the target’s altitude by up to 20 feet in either direction on
your turn. If you are the target, you can move up or down as part of
your move. Otherwise, you can use your action to move the target,
which must remain within the spell’s range. When the spell ends, the
target floats gently to the ground if it is still aloft.
First partial problem with the idea is that slamming the target into walls doesn't work, because the caster can only change altitude.
Falling (PHB 183)
A fall from a great height is one of the most common hazards facing an
adventurer. At the end of a fall, a creature takes 1d6 bludgeoning
damage for every 10 feet it fell, to a maximum of 20d6. The creature
lands prone, unless it avoids taking damage from the fall.
Here is the big problem. Nothing in the description of falling says anything about acceleration, or gravity, or the like.
Instead, it says "At the end of a fall". While a fall is a change in altitude, a change is altitude is not necessarily a fall.
I would rule that while the levitate spell is active, no fall is taking place, and so no falling damage can occur.
Obligatory: 5e is not a physics simulator.