Can I have a character whose class is not primarily based on stealth also frequently and effectively steal? I haven't read the Players Handbook.
Yes, it's possible
While the general answer is YES, it will still rely on your Dexterity ability score and your choices in skills.
Ability scores will be dependent on dice rolls, point buy, or arrays (whichever you use), but skills can be provided by your race, class, or background. Even if you don't choose a race/class combo that has what you're looking for - you may be able to find it in a background (or use a custom background that's approved by your DM.) The Criminal/Spy is a good starting point for deception/stealth, but the PHB's Urchin will give you stealth/sleight of hand.
The bounded accuracy of 5e really shows itself here in skills. This is great because it means lots of different character designs can cross over and while you may not equivalently as good, it's still pretty close.
If being stealthy or having a good sleight of hand is important to your design, I'd recommend increasing your Dexterity and selecting those skill proficiencies.
You can do it frequently, but not necessarily always effectively
But that's not really different from any other class. Being effective is going to depend on the task and the DC set by the DM. You won't necessarily be as effective as a class who has expertise (double proficiency), but you can still be proficient. And even if you're not, with a decent Dex you can still try :)
While succeeding is fun, not succeeding can always open up new story paths.
Spellcasting can boost your abilities as well
In addition to your stats/skills, there is still the option to magically enhance your ability in those areas, either by providing bonuses to your own skills or by utilizing other spell effects to perform those tasks and solve problems. For example, the spell Pass without Trace gives you and nearby allies a +10 bonus to stealth checks, and the spell Passwall lets you create a tunnel that can bypass most physical barriers.
Be aware of consequences
Whether or not you are an expert at thief-type skills, you should always remember that there are consequences to actions. Think about when/where/who during your decision process on performing these skills. If you get caught, you should generally expect a reaction from the game world.