It's not actually an imp
The Find Familiar spell doesn't summon an actual factual cat, bat, imp, pseudodragon, or whatever. When you cast the spell, what you get is:
You gain the service of a familiar, a spirit that takes an animal form you choose
And for the Pact of the Chain warlock, the spell is modified as so:
When you cast the spell, you can choose one of the normal forms for your familiar or one of the following special forms: ...
Nowhere does it say that it changes the fundamental nature of the spell - you don't get an actual imp, you still get a spirit that happens to be taking the form of an imp. The familiar's form reflecting your patron's nature is a stylistic choice that you aren't actually beholden to. The new options don't even override the choice of type that you get from the basic spell, so as written you could easily have a familiar that is in the form of an imp but is actually a fey or celestial spirit.
If you want your familiar to have the imp form for its mechanical benefits, but you don't want it to look quite like an actual imp, it'd seem reasonable to talk to your DM and ask if you can reskin the imp as some sort of tiny celestial servant rather than a little devil, if you want it to reflect a more heavenly patron. If I were your DM, I'd be happy to work with you on the flavour of the class/character in this way - it has no mechanical/balancing impact problems.