Short answer: Never do it on the fly, unless you're experienced.
As you're new to GMing, the best thing you can do right now is prepare. Take all the time you need to find abilities, skills, spells, gear and whatever else you need for your NPC's before the session. Build them in a believable, solid manner (and don't forget to name them all! :D )
As far as deciding how strong they are, and seeing as your question is system-agnostic, the best thing to do, I find, is to give them a challenge, almost equal to how strong they are, as far as numbers go.
Translated in math, a formula would be:
NPC difficulty = PC strength * number of PC's * 0.7
PC strength, of course, is not the common statistic Strength, that measures physical prowess, but the sheer power of one of your Player Characters.
Imagine if you pitted the players against themselves. Now mix your PC's 'evil twin siblings' you just created, so that the PC's can't tell who is who's counterpart (give them unique abilities, skills, etc.) Now make the NPC's you created 30% less efective than the PC's.
This usually works with most games out there. Especially with older games, if you pitted a monster of the same difficulty level as a player character, the player character would promptly die in a horrible manner. The 30% ensures that won't happen, but will still give the players a challenge.
That is the best you can do when preparing. When playing, however, you might discover that the NPC's you designed are too powerful/not powerful enough for the PC's.
The magic word is improvise
If you have the option, don't make all of the NPC's attack at once. Throw a few at the PC's and see how they deal with them, then judge accordingly.
If the NPC's are too strong for the players:
Intervene. Have one of the NPC's fail their spell, or their bow string break, or their sword break. Have them retreat if they see one of their own gravely injured. Make them cowards, stupid and ineffective. It will make up for the overpowered stats.
If the NPC's are not strong enough for the players:
Present 2 of each. Give each one slightly more health. Have a backup list of magical/powerful equipment (depending on the system) in case you need to buff your NPC's equipment. (Give them the items before you let them loose on the PC's, though. Don't change their weapons on the spot)
It's always better if the NPC's are not strong enough, than too strong, and people end up frustrated cause they lost their character they spent so long developing.
In time, you will learn to second-guess your players' reactions and strategies. Watch them deal with an encounter, and remember it for the next time you design a fight. Counter their most common strategies with interesting abilities, items and spells (if your system has magic) and even with terrain, weather conditions and NPC tactics of their own.
Variety is the spice of life
Hope I helped :)