If someone uses sleight of hand and gets noticed, can they still take their movement or swift action before initiative is rolled? Or is initiative rolled as soon as they fail their check? (assuming the offended party wants to attack the would-be-thief)
It works best if initiative is already determined before the attempt is made.
Let me preface by saying: I play rogues a lot, like a lot a lot. Hands down my favorite class. So I have ample experience handling things that they do often. Like picking pockets. This is what my group has found to work best.
Roll initiative before the attempt is even made. If the check is successful, great, our pickpocket walks off without a care. On the other hand, if the check is failed, then that's part of their first turn.
Assuming that the potential mark has no reason to attack our would-be pickpocketer before such an attempt is made, this should be treated as a surprise round for our sticky-fingered friend. They get one Standard or Move Action, and one Swift Action. Since they already spent their standard attempting to pick a pocket (and failing), so they would only have a single swift action left. Then, they likely go again immediately, as our mark looses their first turn from surprise. We then proceed with combat as normal.
Initiative is rolled when the GM says
The rules about Initiative are fairly vague, leaving only this guidance
At the start of a battle, each combatant makes an initiative check.
Fortunately, this is enough to determine that Initiative should not be rolled until one or the other party intends to attack the other (by applying the word "battle" literally) and, at this point, Initiative is started. Both (or all) participants have all their actions to use immediately on their turn, because you were not in Initiative until this point.