When cast on a subject with humanlike sentience, awaken fails
If a creature is the subject of the 4th-level Sor/Wiz spell polymorph [trans] (Player's Handbook 263) or a similar effect that causes a subject that already possesses humanlike sentience to assume animal form therefore the creature type animal yet that also sees the subject retain its humanlike sentience, when the 5th-level Drd spell awaken [trans] (PH 202) is cast on that subject, the awaken spell fails.
That's because there's no distinction between rules text and flavor text in the Player's Handbook like there is in, for example, the Spell Compendium (see this question). With that in mind, the awaken spell, in part, says
You awaken a tree or animal to humanlike sentience. To succeed, you must make a Will save (DC 10 + the animal’s current HD, or the HD the tree will have once awakened). (202)
The Player's Handbook on Spell Failure says
If you ever try to cast a spell in conditions where the characteristics of the spell (range, area, or the like) cannot be made to conform, the casting fails and the spell is wasted. (171)
Because one of the effects the awaken spell is to grant the subject humanlike sentience, the spell fails if it's cast on a creature already possessing an Intelligence score of at least 3 like most PCs. (Yes, even the barbarian.) That is, essentially, one of the first questions the awaken spell asks of its subject is Do you possess humanlike sentience? If the subject's answer is Yes then the spell fails, but, if, instead, the subject's answer is Grr? or rustlerustle then the spell continues.
This means that it doesn't matter if the subject just meets the requirements for the awaken spell's entry Target: Animal or tree touched. The spell's description also mandates that the subject not already possess humanlike sentience. If, for example, a wizard uses polymorph to turn the fighter into a rhino, the fighter—despite possessing the type animal—, still possesses her humanlike sentience, and the awaken spell will fail.
A rules lawyer (and I use the term with affection as a frequent member of that esteemed bar myself) could make a case that the awaken spell does not explicitly specify that the subject can't have humanlike sentience already. However, the spell says that the subject undergoes a change to sentience. If the awaken spell's subject already possesses humanlike sentience there's no change for the subject to undergo or for the spell to make.
The awaken spell doesn't turn a dial but flips a switch. In the case of the awaken spell, there's no room for comparisons (e.g. being more awake after a morning coffee); the spell grants humanlike sentience to a subject that doesn't have it, and to a subject that has it already it does nothing (e.g. being woke by a rooster means needing to go back to sleep before being woke again by an alarm clock… or even the same darn rooster). Attempting to use the awaken spell multiple times to change a subject so that the subject is even more awake is like trying to make something that already is—to even a small degree—become even more pregnant, opaque, ready, or unique.
When cast on a subject without humanlike sentience, awaken succeeds
If a creature is the subject of 5th-level Sor/Wiz spell baleful polymorph [trans] (PH 202-3) or a similar effect that causes a subject that possesses humanlike sentience to assume animal form therefore the creature type animal yet that also sees the subject lose its humanlike sentience, when the 5th-level Drd spell awaken [trans] (PH 202) is cast on that subject, the awaken spell operates normally. (The same applies if the subject is turned into a tree that possesses nonhumanlike sentience, by the way.)
The awaken spell has several conditions for its use, like its entry of Target: Animal or tree touched and that the subject not possess humanlike sentience. If all of the awaken spell's conditions are met, though, the awaken spell does affect the target, and the spell's instantaneous duration means that, once the spells effects occur, those effects remain nonmagically—typically forevermore and immune to nearly all attempts at excising them.
This can be gamed, certainly, by using the baleful polymorph spell, above, and the subject failing both the Fort and Will saves; the 8th-level Sor/Wiz spell polymorph any object [trans] (PH 263) that could be used also to turn the subject into a tree that possesses nonhumanlike sentience; or even the 1st-level Drd spell aquatic escape [trans] (Complete Scoundrel 95) that, as spell of the subschool polymorph, reduces the subject's Intelligence score to that of a toad yet somehow preserves the subject's capacity for understanding language. (Yeah, I don't even.)
Anyway, the effects of the awaken spell on such subjects is brand new each time, not beholden to the Same Effect More than Once in Different Strengths rule (PH 172). That's because the awaken spell's instantaneous duration leaves behind only results rather than an ongoing spell effect. Like in the same way that the next fireball spell doesn't care that you are still damaged by that last fireball spell, the next awaken spell doesn't care that you were affected by a previous awaken spell!
This means that each time such transformed subject also becomes the subject of the awaken spell, the subject will gain another round of awaken results: 2 Hit Dice of the creature type animal or plant, adding +1d3 to its Charisma score, and rerolling its Intelligence score in addition to other effects. For many PCs this will be cause to retire—being, essentially, at least 2 levels ahead but with animal Hit Dice is tough to overcome for a wizard, for example,—, but for some PCs—like those who may have given up on advancing levels anyway—and many NPCs—who would be otherwise stuck forever as level 1 commoners—it may be an incredible boon.