There is no "counter attack" system in Pathfinder as such. First off attacks are presumed to be happening at the same time, and swords, maces, and fists are expected to be flying all the time. The closest thing in Pathfinder is the Attack of Opportunity which models how attacking a character when they have effectively left themselves open, by casting a spell, running past, or some other mechanism.
Let's look at what your suggesting as an attack of opportunity. If you only have it occur on a natural 1, it will happen 5% of the time to all characters regardless of skill. Obviously it benefits characters in melee more as an opportunity to damage and causes a problem as they are taking more damage. It's not going to effect casters or ranged characters much since they aren't making attacks in melee often. It also makes benefits melee characters who have Combat Reflexes (can make more than one Attack of Opportunity) as they would be able to take advantage of it more often. I might also discourage a character from making all their attacks if they have multiple attacks. If you have 3 attacks, and the lowest only has a 10% chance to hit, but also involves a 5% chance of a counterattack you might not take that chance.
In short you will increase the lethality of melee combat slightly across the board, and improve the attractiveness of Combat Reflexes, while slightly decreasing the attractiveness of multiple attacks. It probably wouldn't be game breaking, but it would certainly affect play.
It would also affect the flow of play. Your combats will be interrupted more often with players making out of turn actions more often 5% of the turn for every attack made.