No background features are kept
Magic jar does only what it says it does in its description
Once you possess a creature's body, you control it. Your game statistics are replaced by the statistics of the creature, though you retain your alignment and your Intelligence, Wisdom, and Charisma scores. You retain the benefit of your own class features. If the target has any class levels, you can't use any of its class features.
A spell only does what it says in its description. It very specifically lists what things you keep: Int, Wis, and Cha scores and class features. It does not allow anything else to be transfered.
Other abilities do allow more things
Compare this to the wild shape ability which says:
Your game statistics are replaced by the statistics of the beast, but you retain your alignment, personality, and Intelligence, Wisdom, and Charisma scores. You also retain all of your skill and saving throw proficiencies, in addition to gaining those of the creature.
You retain the benefit of any features from your class, race, or other
source and can use them if the new form is physically capable of doing
so. However, you can’t use any of your special senses, such as
darkvision, unless your new form also has that sense.
It explicitly calls out skill and saving throw proficiencies and race features and a bunch of other things that can be carried over to the new form when you use wild shape. None of these are said in magic jar so none of them carry over to the new body.
Background features are not class features
Background features do not count as class features (they are not listed in the class section of the PHB, they are listed in the Backgrounds section).
Class-granted proficiencies are kept
Proficiencies are listed under "Class Features" in the PHB. So, any proficiencies that you got from your class (and class only) would be able to be carried over to the new body. Proficiencies granted for any other reason (feat, background, race) are not transferred.
Thus you do not get to keep background features in the new creature, but you are able to keep class-granted proficiencies.