On Abstract Damage
under no circumstances should you be able to damage the armor wearer unless the attack is made at a not armored part of the body.
Vampire, like many RPGs uses an abstract damage system. This means that a single attack/defense/damage/soak series of rolls represents a much more complex series of actions than a single strike and parry.
For example:
You stab at him with your knife, but he ducks and grabs your knife arm. You continue to push the knife towards him, matching your force against his, but he's able to gradually twist your arm away from him. You break the stalemate with a swift knee to the gut. As he doubles over, you drive your knife into the unprotected area behind his armpit.
In system terms, most of the time, a scene like this is handled with a simple roll.
Specifically, most of the time, the location you hit someone is unspecified. So unless the armor wearer is literally encased in a magical form of flexible steel, there are going to be chinks and other weak spots that can be exploited.
Better armor has fewer chinks, which is represented with a higher armor rating. This makes it more difficult to damage someone wearing the armor, but not impossible. This is why damage rolls are less likely to inflict damage on someone wearing good armor, but damage can still be inflicted.
On Bricks
The key flaw to your argument is that combat in the dark ages is rarely going to be about plastic vs. bricks, or steel vs. bricks. It's almost always going to be steel on steel. At that point, the important aspects are force, angle of attacks, and the shape of the weapons involved (a slash is much less likely to pierce chain mail than a thrust is).
Even when it's steel vs. wood, or steel vs. iron, combat training is all about controlling the angles of impact. A steel sword doesn't immediately sunder a wooden shield because of the angles of deflection involved.
There's also the bludgeoning issue: A mace might not actually break a helmet, but that doesn't mean your brain hasn't just been scrambled.
On House Rules
The main issue I see with your rule is that it breaks down into two scenarios:
The weapon is inadequate to pierce the armor, in which case the armor wearer is invulnerable.
The weapon is adequate to pierce the armor, in which case the armor degrades severely in combat.
The first issue causes severe gameplay problems. Invulnerable PCs make combat a foregone conclusion (you will win, given enough grinding). Invulnerable NPCs make for comedic and frustrating fights (well, we can't hurt him, but we can mess with him until we get bored).
The second can start to make armor feel disposable. If your armor is torn to ribbons every third fight, it doesn't feel very durable.
Were I wanting to add more emphasis to armor, I'd do it this way:
Move most of the damage levels from the character to the armor. Characters without armor die very quickly.
Damaged armor provides less protection over time, making it more likely that the character dies. However, penalties start very minor / progress slowly at low levels.
Damaged armor below a certain threshold is repaired simply by removing it, performing basic maintenance, and reequiping it. Only serious damage requires repair. Destruction is either unheard of, or only happens after massive damage.
On Armor in Other Systems
Other systems take a different tactic: If the character is damaged, then that must mean the attacker damaged the armor in some way. Whenever the character takes damage, damage/degrade the armor.