Spiders are ambush predators, and would use the 5e hiding rules to do this
To go over your example:
A goblin hides in a bush, and a player must make a passive perception check to notice them, when they are within the vision of the player. Or they can make an active perception check to search for them.
Sort of. Players don't make passive perception checks, their passive perception is always on (it's a passive ability, not an active ability). If the DC to notice a hidden creature is a 12, and one of your PCs has a 13 passive perception, they will detect the hidden creature whether or not they're looking for it.
In my adventure, I had a spider hiding in the fog. The fog was so thick that vision was only 5 ft. If a spider is 20ft away presumably you cannot see it, either passively or by making a perception check, as it's no different to it being behind a concrete wall in terms of sight.
Keep in mind that perception is not only visual. While the players may not be able to see into the fog, they can hear the spider clicking around or smell it (I don't know if spiders smell bad or not). So while you are correct that they won't be able to see it, they may well be able to perceive it.
RAW, here is how you deal with the heavy fog:
A heavily obscured area—such as darkness, opaque
fog, or dense foliage—blocks vision entirely. A creature
in a heavily obscured area effectively suffers from the
blinded condition (see appendix A). A heavily obscured area doesn’t blind you, but you are effectively blinded when you try to see something obscured by it
This takes us to the 'Blinded' condition:
A blinded creature can’t see and automatically fails
any ability check that requires sight. Attack rolls against the creature have advantage, and the creature’s attack rolls have disadvantage.
So if the spider is 100% silent and has no smell, then you could reasonably assume that a sight check is required to perceive it. In that case, yes, it would be impossible for your PCs to detect it. But, very few things in this world are completely 100% silent, so RAW there is probably a change (albeit a very small one) that the PCs could detect it with an extremely good active perception roll (assuming they had reason to make one, but thick fog in a cave sounds like a good reason to me).
So how do you handle stealth in this instance? Because if a player moves within 5ft of the spider, it will see it automatically as it won't be behind cover anymore.
The details are up to you, but you can either determine that the spider is unhearable and unperceivable and as such has 100% perfect stealth, or that it may be perceivable and is potentially vulnerable to a great perception check.
To answer your specific question about how the combat would likely play out and what the spider itself would do
So can the spider actually hide? If it did hide using its stealth ability, how would you explain that to the players? The players won't be able to swing at where they think the spider is, because the players range is 5ft and they can see 5ft, so there would be no guesswork involved.
Looking at the relevant rules for hiding:
You can’t hide from a creature that can see you, and if you
make noise (such as shouting a warning or knocking over a
vase), you give away your position. An invisible creature can’t
be seen, so it can always try to hide.
So, your spider seems to fit the bill as a hide-able creature in a hide-able situation, so your spider can take the hide action, make his or her stealth check, and be hidden from the PCs until they detect it.
Obviously I want the spider to use the fog to its advantage, as it has 10ft of blindsight so can see 5ft further than the players.
RAW, that makes perfect sense. Just keep in mind that if your PCs are being quiet and are more than 10 feet from the spider, then the spider needs to make an effort to detect them as well. Basically, if both groups know there is danger in the fog, then they'll both be likely trying to hide from each other until first contact is made.
I considered it hiding to mean that it is carefully tiptoeing around the players, staying out of view but watching them so if a player does move within 5ft in technical game terms, it still can't see the spider because although the spider technically occupies the adjacent square, it is hiding somewhere else.
Hiding has a very specific meaning in 5e. When you're hidden, it means that you have a standing stealth score than needs to be beaten by either a passive or active perception check. As long as it is not beaten, and you don't make an attack, you are hidden. Once it is beaten or you attack, you are no longer hidden. An ambush predator like a spider would likely use this to get into position around the PCs (using its blindsight to stay 10 feet away but still detect the players), and then strike when one gets separated. Once it strikes, it has attacked and loses its 'hidden' status. After the attack, it can attempt to move away from all the PCs (to once again gain the heavily obscured condition so that it can hide at will). Then, it will take its next action to hide and move away from the PCs. It will roll another stealth check and it will use that to contest the PCs active and passive perception.
Keep in mind that as long as the PCs are in the heavy fog, they will be unable to see the spider until it gets next to them. Thus, they will have disadvantage to all attacks against it, and it will have advantage on all attacks against them. Also keep in mind that fog is generally dispersed with a wind of moderate or stronger intensity, so if one of your PCs has a wind spell they should be able to disperse the fog.
