While I mostly agree on MonkeezOnFire's answer, I would argue that the first scenario would work if the rust was scratched off (which you say has been done), and if the length or width of the hammer is below 1 ft. This allows the spell to be flexible if you're creative, but it wouldn't work on all situations. In general, most weapons are longer than a foot in at least one dimension. If you're afraid it's still OP, it's worth noting the spell is transmutation, not conjuration, so it does not summon any new materials, (flesh related transmutation spells are complicated though, let's ignore those for now) so scratching off rust reduces the volume of the hammer, making it smaller and slightly deformed. If it did summon materials, it would be possible to rip off pieces of an item to sell, before mending them again. These kinds of tricks are fine only if the amount of times it can be done is limited (i.e. Not a cantrip or orison)
Or you can go the traditional jerk-but-fair-and-honest DM answer and say a scratched hammer has hundreds breaks or tears, requiring a whole hour or two of casting the same orison until the cleric's eyes fall off. Or a scratch is not considered a tear or break, for some odd English-related reason.
TLDR; Doesn't work on most weapons, of the rust covers the whole thing, but be creative! Or you can say a thousand scratches cover the hammer, requiring a thousand casts.