RAW, movement options are limited when it is not your turn
What does Dissonant Whispers do?
The target must make a Wisdom saving throw. On a failed save, it takes 3d6 psychic damage and must immediately use its reaction, if available, to move as far as its speed allows away from you. The creature doesn’t move into obviously dangerous ground, such as a fire or a pit.
In a natural world (that is, one not governed by the abstractions of simplified game rules), a mounted target that was forced into moving away from the source of its pain could ride its mount away, could dismount and move away, or could fall off the mount and then move away. Within the mechanics of 5e, however, these options are constrained by the rules regarding movement. The most important of these constraints is the fact that the target is not moving on its own turn.
In general, you cannot move when it is not your turn. If something permits you to move when it is not your turn, that thing itself must then spell out the conditions under which you may move, as an example of specific over general.
A creature mounted on another creature and controlling it, can also 'direct' the movement of the mount, but since your controlled mount moves on your initiative, this option applies only on your own turn.
Thus, if the knight is being affected by dissonant whispers on the turn of the caster, the knight can move only in the way that the spell itself explicitly permits. Since the spell does not explicitly allow you to direct a mount...
The knight cannot ride the mount away in response to the spell.
If the knight cannot ride away, can it then dismount before fleeing?
Dismounting is a game term, and is governed by the 'Mounting and Dismounting' rules in the Mounted Combat section, which (emphasis mine) say:
Once during your move, you can mount a creature that is within 5 feet of you or dismount. Doing so costs an amount of movement equal to half your speed.
What does "during your move" mean? I believe that a strict RAW approach suggests that "your move" is any movement that is undertaken on your turn, as described in 'Breaking up your move' (emphases mine):
You can break up your movement on your turn, using some of your speed before and after your action. For example, if you have a speed of 30 feet, you can move 10 feet, take your action, and then move 20 feet.
If the rules had intended to allow mounting or dismounting during forced movement, then rather than saying you may dismount once during your move, they would have said once when you move, while you are moving, or during a move.
[Note that my interpretation that 'your move' is movement that occurs on 'your turn' is not universally acknowledged. Dale M, for example, believes that 'your move' is any movement you make, even forced movement, as he says in the chat discussion for this answer and others. The rest of my answer here presupposes that your move is only movement you make on your turn]
Since dismounting is a movement option that may be undertaken only on your own turn, a knight affected by a spell on the turn of the caster may not dismount unless the spell itself explicitly permits dismounting. Dissonant whispers does not.
The knight cannot dismount the mount in response to the spell.
If the knight can neither ride the mount away nor dismount it, can the knight simply fall off the mount and then move away?
Suppose the afflicted knight moves off the mount without dismounting. This movement is explicitly permitted by the spell. Once the knight has moved out of the square of the mount, being unsupported, the knight would then fall to the ground. Note that falling is not a form of movement that you elect; rather, it is something that happens to you, and thus can occur regardless of whether or not it is your turn.
Further, falling from a mount has the consequence of "landing prone in a space within 5 feet of it" (note that this is a specific exception to the general falling rules, which state that you do not land prone if you have somehow avoided damage from a fall, so even if your fall from the mount does no damage, you still land prone).
The RAW response to dissonant whispers is for the knight to fall off the mount, landing prone in a square adjacent to the mount but away from the caster, and then continue their movement away.
Once the knight is off the mount and prone on the ground, the spell dictates that they must continue their movement away from the caster, as far as their speed allows. They can now do one of two things. They may crawl away from the caster, with each foot of crawling costing two feet of movement. Or, they may stand up at a cost of half their speed and then continue to move away from the caster.
One might reasonably question whether the knight can stand up if it is not their turn, given that they cannot, for example, dismount. The rules for standing up say:
Standing up takes more effort; doing so costs an amount of movement equal to half your speed. For example, if your speed is 30 feet, you must spend 15 feet of movement to stand up. You can't stand up if you don't have enough movement left or if your speed is 0.
Note that standing up does not take your movement, it simply requires an amount of movement. Dismounting happens only during your move (on your turn), but standing up is not so limited. Standing up requires only that you have movement, and in this case that movement is supplied by the dissonant whispers spell; it is not your movement, but it is movement you are permitted to spend.
Once the knight affected by dissonant whispers has fallen off their mount, they may then either crawl away, or stand up and move away, until they have exhausted the movement provided by the spell.