You can put it in, but it might not stick
In summary: you can finagle the will-o'-wisp into the bag, but whether you can keep it in there, or whether it can escape or destroy the bag from the inside will be up to your DM. Either of the two would make the effort a bad idea in most circumstances.
Can you get it in?
The Will-o'-Wisp is immune to the grappled condition, so you cannot grapple it to put it in the bag, but you still could push it in. To paraphrase from that answer
There are no general rules for incorporeal creatures that make them entirely immune to physical interactions, the Will-o'-Wisp only has resistance to physical damage, so it clearly can be interacted with.
The next question is if the bag allows this. You clearly can put creatures into the bag, as the bag states:
Breathing creatures inside the bag can survive up to a number of minutes equal to 10 divided by the number of creatures (minimum 1 minute), after which time they begin to suffocate.
The bag has no special rules stipulations for putting something into the bag, so that works like with any old bag. If the bag is open, you should be able to push the will-o'-wisp in, or maybe can try and snatch it with the opened bag.
Can you keep it in?
The interior of the bag is an extradimensional space. That means there is no other side of the interior of the bag. So there is nowhere to go to for the will-o'wisp, once it is in the bag, other than maybe the lid, even though it could move through walls. Can it get back out through the lid?
The bag states that
Retrieving an item from the bag requires an action.
The overwhelming consensus is that you cannot open a bag of holding from the inside. Even though the will-o'-wisp can fly, and is incorporeal so it could move through the lid without opening it, the answer's interpretation of the sentence quoted above is that the only way to get something out of the bag is to retrieve it.
I don't agree: that you need an action to retrieve an item does not logically imply that a creature cannot leave it actively. You are not retrieving this creature when it does that, so the requirement to use your external action does not apply, and the wisp could just move out through the closed lid. But I think that there are such different viewpoints means this is best left to your DM do decide.
Can it destroy the bag?
Furthermore, there is no conclusion if you can damage the bag from the inside, because the bag's text does not talk about that either. It merely says
If the bag is overloaded, pierced, or torn, it ruptures and is destroyed, and its contents are scattered in the Astral Plane.
You could rule that because the inside is an extradimensional space this does not work, but earlier editions of D&D allowed destroying it from the inside. So that again is up to the DM.
In addition, the will-o'-wisp has no attack that would allow it to pierce or tear the bag, it can only shock it with its shock attack. So rules as written it cannot destroy the bag from the inside, even if that is generally possible, and it again would be a DM call if burning a hole with electricity would work to rupture the bag, as this upvoted answer postulates. If it would, the will-o'wisp could escape to the astral plane, destroying your bag and getting rid of all your stuff while doing so.