It needs to be one near the top
From the PHB p.109:
Fiends powerful enough to
forge a pact include demon lords such as Demogorgon,
Orcus, Fraz’Urb-luu, and Baphomet; archdevils such
as Asmodeus, Dispater, Mephistopheles, and Belial;
pit fiends and balors that are especially mighty; and
ultroloths and other lords of the yugoloths.
Your world, your rules
If you don't like that, change it. If you think it would be fun for the pact to be held by an imp or rakshasa then do it. Just think through the implications - a warlock without a patron isn't a warlock - I can see a lot of stress on a warlock with a weak patron trying to keep it alive.
Notwithstanding, your warlock is not important enough to deal with the boss
I have a contract with Microsoft - when Microsoft deals with me I don't talk to the CEO. I can certainly see devils running hell as a multi-planar corporation. Sure your contract is with Asmodeus but your case manager is Roger, imp 3rd class who has Thursdays and Saturdays off and will get back to you within 10 business days after you lodge the correct paperwork. You can, of course, call customer support to hear "We are currently dealing with other warlocks. Your pact is important to us so please hold the line. Current wait time is approximately 342 years. Thank you."
Demons of course do things far more chaotically. Yes, your pact is with Fraz’Urb-luu whose filing system consists of dropping things wherever he happens to be and then moving on. The guys you deal with are never consistent depending on who happened to take an interest at the time, assuming, of course, that someone bothered. You never know if any given demon who says they are working for Fraz’Urb-luu actually is or isn't or if Fraz’Urb-luu wants them to do whatever they want you to do or not. You wanted a quiet life? Don't go making deals with the most chaotic and evil beings in the multiverse.
Lords of the yugoloths are more likely to give the personal touch. For them, corrupting mortals to evil is neither a bureaucratic process nor something to do when they feel like it. Each corrupted mortal is a work of art to be admired for all eternity and they constantly strive for perfection in that art. Fashions change of course, this millennia the fashion is to corrupt as quickly as possible, a soul corrupted in a day is better than one that takes a week. Next millennia, the slow burn is admired, slowly stripping away everything and everyone the mortal cares about until the sink, at the end of their life, into the depths of evil despair - more satisfying than a cold beer on a hot day.
I can also see a secondary market in warlock pacts (run by devils, of course) where warlock pacts, and options and derivatives of them, are traded back and forth. Perhaps angels have to come to buy out contracts of warlocks who died in a state of grace?
