The rules imply that objects have two different "modes" of existence. They can be independent objects, subject to targeting and damage from area-effect spells. Or, if carried by a character, they become "attributes" or "parts" of that character, and no more subject to specific damage than, say, the character's ears. These are not mutually exclusive - an object can be carried by a character and still be targeted by a spell or action. But they cannot be both at once in the context of a single action.
Note that it is the attacker, not the character, that determines which "mode" applies. The attacker indicates this by specifying what he is "targeting".
In the OP example, the objects carried by the victim would not be subject to the spell effects unless either 1) the caster targeted the specific item or 2) the spell specifically indicates that carried items are affected.
For Banishment (or Shove), since the carried items were not specifically targeted, they are considered to be part of the character and are banished with the character. There is nothing weird about this. After all, hair is nothing more than dead keratin that happens to still be attached to the skin, yet we would consider it even weirder if Banishment left the victim with a bald pate and silky-smooth legs. We intuitively understand that a character is more than just his living tissue, and the game makes no distinction between items that are carried in a pouch, gripped tightly in the hand, or attached like hair to the body.
The Pinocchio Paradox
There is one strange glitch in the game's handling of objects which I call the "Pinocchio Paradox". Suppose Geppetto is playing with his marionette when his archenemy strikes him with an area-effect spell. The puppet, being "carried" by Geppetto, is unharmed. On the next turn, the Blue Fairy turns the puppet into a real boy, and then Geppetto is struck with the same area-effect spell. This time, Pinocchio is caught in the spell and takes damage.
Although at first this seems like a hole in the rules, it really isn't. The essential distinction between the puppet and the boy is that the boy can take independent action. Damage in the game is only recorded when it has a game effect - that is, it affects the types of actions possible and the probability of specific results. On the first round, of course the puppet was damaged in the colloquial sense - the DM might describe the charring of the wood or the tangling of the strings - but that damage has no impact on combat and therefore is not recorded. On the second round, the damage done to the boy does have an impact on combat, as the boy has the potential for independent action which he may or may not now be able to (or wish to) take advantage of.
You can further imagine that the puppet could be magically (or mechanically) animated to make the situation seem even more paradoxical, but the rule for resolving it does not change. You can also consider the opposite paradox - is a baby being carried count as an object or a character? There is a simple test for these situations - is the object (potentially) capable of taking Actions and is therefore a "character"? If not, any damage (unless the object is specifically targeted) is considered cosmetic and for simplicity's sake is not recorded.