What’s happening is that, with Two-Weapon Fighting, you have a −4 penalty (note that this is only −2 if you have a Light weapon in your offhand) on all attacks. When you have Improved Two-Weapon Fighting, you get a second bonus attack at −5, because effectively your iterative is now getting a bonus offhand attack.
So, if you have BAB +6/+1, and only Two-Weapon Fighting, and you are using a Light weapon in the offhand, your attack routine is this:
Main hand +4, offhand +4, iterative −1
With Improved Two-Weapon Fighting, that iterative also gets a bonus attack, so you have this:
Main hand +4, offhand +4, 1st main iterative −1, 2nd offhand iterative −1
If you have a one-handed weapon in your offhand for some reason, subtract an additional −2 from these, since in that case the TWF penalty is −4. You should basically never do that, though.