They can be whomever you need them to be
The Sword Coast Adventurer's Guide is your best bet for information on that area (p. 9 for Amn), and it doesn't go into much detail, but there are a few parts which could be of use:
[...] Amn is a place where the wealthy rule, openly
and without pretense. Shrewd traders and ruthless in
business, Amnians believe that the end of a successful
transaction is justified by any means, ethical or otherwise.
[...] The oligarchs utterly control their nation,
Which certainly can support conscription or possibly coerced service. Otherwise you could base yourself on
So pervasive is the sway of Amn's oligarchy that
few crimes merit physical punishment but those that
involve the use of arcane magic or an offense against
one of the council's merchant houses. Other infractions
are forgiven after the miscreant makes payment of an
And add that failure or inability to pay would mean a punishment of service and thus fill your cave with anyone with a debt (merchants, guildmembers etc.) and/or by people who have come to commit a crime.
Specifically in Sword Coast Adventurer's Guide (p.44) is included:
NOTE TO THE DM: MAKING THE REALMS YOURS
The information in this chapter is intentionally nonspecific
with respect to threats, monsters, and statistics. While
you can use these details to flavor your descriptions of the
towns and cities your players might visit, you should in no
way consider these pieces of narration definitive. They're
meant to provide touchstones, not to constrain your players
to a narrow conception of the world.
You might decide to change some of the details, either
to surprise your players or to account for events that have
transpired since they obtained the information herein.
You're encouraged to take what's presented here and make
it, and the Forgotten Realms, your own.
While it doesn't specifically mention your problem, it does give the reason for why (5ed) D&D lore doesn't specify such things. Which is that you are empowered to choose whatever fits best for your worlds and the story you would like to tell.
(Much of these sentiments are repeated in the first chapter of the DMG: A World of Your Own)