In 5e, you can only make opportunity attacks against targets you can see.
I can't find similar language in 3.5, but the rules are a bit scattered.
Can an invisible attacker run circles around you without any problems?
Role-playing Games Stack Exchange is a question and answer site for gamemasters and players of tabletop, paper-and-pencil role-playing games. It only takes a minute to sign up.Sign up to join this community
You can't attack a target with total concealment
The rules for Total concealment (PH p152) state (emphasis mine):
If you have line of effect to a target but not line of sight (for instance, if he is in total darkness or invisible or if you're blinded) he is considered to have total concealment from you...
You can't execute an attack of opportunity against an opponent with total concealment even if you know what square or squares the opponent occupies.
So an invisible opponent cannot be subjected to attacks of opportunity unless you have a way to see through the invisibility.