The rules for a Long Rest (PHB p. 186) state the following:
A long rest is a period of extended downtime, at least 8 hours long, during which a character sleeps or perform slight activity: reading, talking, eating, or standing watch for no more than 2 hours. If the rest is interrupted by a period of strenuous activity—at least 1 hour of walking, fighting, casting spells, or similar adventuring activity—the characters must begin the rest again to gain any benefit from it.
Right there are two restrictions that can prevent a player from finishing a long rest. Let's look at each one and how it could apply.
... or standing watch for no more than 2 hours.
This could become relevant if the character is terrified of his or her captors. They would be constantly on the lookout from them coming back, and as such would be on edge for more than 2 hours at a time.
In the situation that the player would not be on the lookout for his/her captors, he/she would eventually need to sleep. Unfortunately, the PHB does not have rules on sleeping. Xanathar's Guide to Everything does however (p. 78).
A long rest is never mandatory, but going without sleep does have its consequences. If you want to account for the effects of sleep deprivation on characters and creatures, use these rules.
Whenever you end a 24-hour period without finishing a long rest, you must succeed on a DC 10 Constitution saving throw or suffer one level of exhaustion.
It becomes harder to fight off exhaustion if you stay awake for multiple days. After the first 24 hours, the DC increases by 5 for each consecutive 24-hour period without a long rest. The DC resets to 10 when you finish a long rest.
If the player tried to keep staying awake, eventually they would rack up enough levels of exhaustion that they would just die. However, chances are they would pass out before then. That part is not technically RAW, but going from being extremely tired to dying is a bit of a leap.
If the rest is interrupted by a period of strenuous activity—at least 1 hour of walking, fighting, casting spells, or similar adventuring activity—the characters must begin the rest again to gain any benefit from it.
This is how the captors would go about stopping a rest. Personally, I think things like having to balance, stand in one place, and things of that nature are all physically tiring, and as such would qualify as stopping a long-rest.
The captors could do something like put the player over a pool of ice water on a tiny platform that they have to actively stay on or fall into the water. They could do what you suggest and put them in an Iron-Maiden-esque device that would prevent the character from leaning against it.
As long as the character has to do something strenuous, they can't take a long rest or sleep. This is everything that affect how resting or sleeping work (to my knowledge).