I don't have a 5th edition DMG on hand, but I recall that in third edition, the DMG says that, on average, an encounter should consume 20% of the parties resources. Did that happen? If so, then it qualifies as a full encounter regardless of how much combat occurred. If not, then you might consider giving them a proportion of XP based on this (e.g. if it used 10% of their resources, then give them half XP), or you might want to categorize it as a roleplay encounter instead - in which case you give them as much XP as you feel is appropriate, with no justification required.
While this is based on the rules as written for a previous edition, I think that the fundamental basis of it is still applicable - in whatever edition of D&D you are using, you don't want your players to easily breeze through with no difficulty, but at the same time you don't want them to overwhelmed. So the rules were set up so that any given encounter required 20% of their resources to overcome, which is a much better absolute measure of difficulty than the challenge rating, and which the challenge rating is designed to approximate.
In any given game, the party composition and equipment of your players will vary, meaning that some CR-appropriate enemies will be harder and some will be easier, but it would average out in the end to about 20% per encounter. The same applies to non-combat encounters, which of course includes both roleplay, but also stealth missions like your example.
To put it more mathematically: Let A by the amount of resources needed to defeat your bandits, and B the amount of resources they actually used. Then let X be the amount of XP they would get for beating the bandits. You should award them X * B/A XP.
You might also want to consider using play time as a stand-in for resources. i.e. if you want them to advance in 1 level every play session, and they used 20% of the session avoiding the bandits, then give them 20% of the XP they need to level. This requires a bit more planning to pull of, but is an equally reasonable way to do things, and means you don't need to worry too much about calculating exact xp for the encounter. It depends on what kind of game style you are running - after all, you are the DM, and so for you the rules are really more like guidelines...