There's no specific rule or best way. Roll for Shoes is malleable, and you'll work out what works for you.
I have a couple of preferred approaches I've used, which work very well for setting up the number of dice to use for the difficulty of the task. The first approach is complex, the second is less so, and they can be used together.
There is not anything saying you're supposed to oppose with the same number of dice. That's certainly a valid approach, but as you've noticed, it's weird and kinda guarantees the players always have a 50% chance of success — or slightly less than, because they have to beat the opposing roll.
Build up characters to serve as opposing forces
This is high complexity, and mostly suitable for longer games that'll go for an hour or more.
As the GM, you may want to provide opposition in the form of characters with actual skills to use: when the fortress guard NPCs have Listen closely 3, you know exactly how many dice to roll.
I say opposing forces because people aren't the only opposition you might provide. You may assign yourself, the game world, factions, groups, or individuals, or yet other things. Roll for Shoes is a game where mostly anything could get modelled as a character if it seems reasonably appropriate for it to have its own skill set.
My own tendency is to introduce each or any of those - typically The World, a major NPC or two, or a major faction or two - when I decide I'd like to have them there for the players to interact with. I'll start them off with a small number of skills (2 or 3 will often do, they can branch out later) suited to their purpose, with scores set at points suitable for challenging or working with my players. (Possibly challenging them a lot if I want to.)
Passive difficulty rolls from nothing
This is fairly simple, and more suitable for short games where it's not worth building up characters as opposition, or for spontaneous opposition in longer games.
Sometimes you just want to know how tough something is. The character stuff can be a lot to mess around with.
So, for just determining passively how difficult something is, do this:
- Put two dice down on the table where people can see them. (If you're in the late game and scores are very high, you might want to put down a third.)
- Suggest each relevant factor that might make this task easier or harder for the character.
- Put another die down for each thing that'd make it harder.
- Take one away for each thing that'd make it easier.
- Do this right in front of them one by one so they can watch.
- The players can suggest the same.
- Each thing should get some semblance of agreement, or at least, not objection. You're here to have fun, make sure people are content with the results.
At the end, you wind up with a row of dice - maybe a long one, maybe just one. Roll them.