Ruling a fief in absentia happened throughout recorded history.
A 'Seneschal' or 'Reeve' or 'Housecarl' or any number of other names describes the position of 'someone ruling a noble's lands while he isn't there'. Perhaps he's in the capital, politicking, or partying, or he's off crusading, fighting a war, trading, socializing with a neighbour, or just traveling.
Often these duties were also passed off to another family member, a younger brother, or cousin. Sometimes a noble owned so many estates they were physically unable to even oversee all of them.
How to Portray this: When the PCs return from their adventure after being called out by the king to do blah, which as Landed Nobles they cannot refuse, up bustles Ridley, their seneschal, with an armfull of reports, talking about the new orchard's production and a marriage in the local village which he decided to break open the stores to create an impromptu celebration etc, and your PCs can either brush him off and go drinking, or actually get involved in the village life through him - he brings them problems to rule on, options about whether to build a new mill now or keep carting the wheat to Highdale, etc.
But why would the peasants still consider them their lords?: Everyone in medieval times was keenly aware of their social status. If you kill a noble and then took his lands, every single other noble, and the king, and the churches, and the common folk, would come and murder you. If you did it with a writ from that noble's liege (or the king) saying he is a traitor? Or people could pretend you did afterwards? Different story. But other than 'not staying on good terms with the other nobles', there is no way to remove a noble from his or her position.
In the Iron Age, if a King sailed off and his retainers didn't retain their loyalty, he'd come back and someone else would be ruling there and he would now need to reclaim his throne, even if just by walking in with his now battle hardened warhost and saying 'you think you're tough enough'. In the Feudal Age, however, there are rules. And the rules are unspoken and basically based on what other people of your social class think of you, but if you break those rules, the entire society comes along in rank and pageantry and murders the hell out of you.
So if the PCs' land gets taken off them, either their liege lord did it, the King did it, a highly skilled Rogue did it, or it's the goddamn Nazgul, because no-one else would have the power to stand up to the entirety of France and say 'go die in a fire'.
Making Rulership meaningful
Social - Nobles treat other nobles completely differently to how they treat random mercenaries they are sending to kill a griffin. Kingdom politics (Vampire: The Masquerade has some good advice on Dark Age-esque politics and how to portray it) are now something your PCs can take part in without necessarily needing to create or defeat a revolution. Maneuvouering to oust their neighbour and take over his lands, get the ear of the king or a greater noble, frame someone who has taken a dislike to them, etc etc, normal social campaign stuff but with realm-wide consequences that you can show, even to the point of hanging plot elements on it, such as exposing someone as a secret cultist or the PCs actions bringing a border conflict into a simmering near-war that they then have to avert.
Fiscal - Using the PCs lands as a gold-sink is a bad idea. Using the PCs lands as a gold sink is a bad idea. To make choices meaningful, they should be able to rip gold out of their lands to use for stuff, or put gold in to make their lands better, and both choices should be reflected by the state of their lands and the amount of sidequests they have to do to stop peasant revolts/banditry/rival nobles stealing stuff from poorly defended lands/slavers/etc. Additionally, if they have good lands they should get free hirelings in the form of local knights swearing allegiance with the lands providing the pay, peasants existing in enough numbers to sign on as armsmen, etc, and in extremity the land should be worth more as a bartering chip in high-noble negotiations. A estate worth 3 million pounds, or ripping 100,000 pounds from it in cruel taxes and lowering it's worth to roughly 500k, one of those estates is worth more when you're trying to buy the Duke of Somerset around to bring his army to Avalon and stop the Dark Lord Sauron from forging the One True Ring.