Would a Goat with a Gore attack be considered armed for the purposes of the rules outlined by this feat? If it was somehow unable to attack with a Gore and tried to Bite would it need Improved Unarmed Strike to avoid an Attack of Opportunity? Per these rules would the Bite or Kick (hooves) deal non-lethal damage?
No. Unarmed strikes and natural attacks are separate, distinct things, and the rules for one do not apply to the rules for the other. And gore is a type of natural attack.
A short list of some of the differences:
Natural attacks never get iteratives; unarmed strikes do.
You can use natural attacks you have once each in a full-attack; you can only use unarmed strikes if you have attacks (from BAB, two-weapon fighting, things like haste, or whatever) to use for them. For example, if a creature has four arms, that creature can automatically attack four times if those arms end in claws, but if they end in fists that creature can only attack four times by using up four attacks, e.g. from iteratives or by using Multiweapon Fighting.
Unarmed strikes are monk weapons; natural attacks are not.
Unarmed strikes require proficiency and (to avoid an AoO) Improved Unarmed Strike. With natural attacks, however, a creature is always proficient and never provokes an attack of opportunity for using them.
Note that Feral Combat Training muddies the water here somewhat, since it makes natural attacks count as monk weapons and allows them to be used with a flurry of blows, so that they do get iteratives. But that’s a special case.
Anyway, it would be too much for an answer here to reproduce the entire natural attack rules, so have a link.
A goat that wants to do more than just gore can take Improved Unarmed Strike, but it's a little more complicated than that
The typical goat has but a lone natural attack, its bite. As a creature possessing the type animal, a goat is considered proficient with its gore and pretty much armed with its gore all the time, therefore threatening an area with its gore, not provoking attacks of opportunity from the defender when it makes a gore attack, and so on, it and its gore being treated by the game much like a typical creature armed with a melee weapon.
However, a goat—or just about any other creature, really—can opt to employ an unarmed strike either instead of or in addition to its natural attacks. In the latter case, its unarmed strike is considered an attack with a weapon and subsequently makes all the goat's natural attacks into secondary attacks—this is probably the simplest explanation of that interaction.
The goat's unarmed strike could very well be described as the goat delivering a painful but nonlethal bite (or even a lethal bite if the goat is willing to suffer a −4 penalty on its unarmed strike attack roll), but it needn't be and could just as easily be described as a kick or a goat-style body check.
No matter what though, a goat that opts to use a weapon other than its gore—and that includes its unarmed strike—will suffer a −4 penalty on the attack roll with that weapon unless it's proficient with that weapon. (The unarmed strike is a simple weapon.) Further, the goat will probably want the feat Improved Unarmed Strike so as to not provoke the attack of opportunity that an unarmed strike usually provokes from the target of the attack.