As per RAW:
Accidentally Ending Movement in an Illegal Space:
Sometimes a character ends its movement while moving through a space
where it’s not allowed to stop. When that happens, put your miniature
in the last legal position you occupied, or the closest legal
position, if there’s a legal position that’s closer.
Source: Special Movement Rules
So I think that 5ft movement will not happen. Your DM will probably justify that the archer will feel some kind of force pushing him back, and he will then know there is something in that square (total concealment for attacks still applies). Also, you're not entitled to AoO, because a 5ft step doesn't provoke them.
Even if RAW is kinda clear in this case, your DM may rule otherwise. One possible resolution for this action would be the archer initiating an accidental Overrun combat manuever, with some kind of penality.
As a standard action, taken during your move or as part of a charge,
you can attempt to overrun your target, moving through its square. You
can only overrun an opponent who is no more than one size category
larger than you. If you do not have the Improved Overrun feat, or a
similar ability, initiating an overrun provokes an attack of
opportunity from the target of your maneuver. If your overrun attempt
fails, you stop in the space directly in front of the opponent, or the
nearest open space in front of the creature if there are other
creatures occupying that space.
When you attempt to overrun a target, it can choose to avoid you,
allowing you to pass through its square without requiring an attack.
If your target does not avoid you, make a combat maneuver check as
normal. If your maneuver is successful, you move through the target's
space. If your attack exceeds your opponent's CMD by 5 or more, you
move through the target's space and the target is knocked prone. If
the target has more than two legs, add +2 to the DC of the combat
maneuver attack roll for each additional leg it has.
The archer would then roll CMB, and receive attacks of opportunity in the process. This may better reflect the situation.
EDIT: Added sources, as per comments request