Terms, first. I'm going to refer to the character that places a mark as the Marker and the target of the mark as the Marked. This might get a little confusing/redundant because they're so similar, but bear with me.
Second, mechanical considerations aren't the focus of this. It doesn't sound like you're having trouble with the mechanics of a mark, just what they represent in roleplay.
Placing a mark is the representation of the Marker focusing their attention on the Marked character. Regardless of how they do it, the Marker puts themselves in a position to "punish" the Marked if they are ignored and exploit even the smallest distraction to interfere with the Marked's attacks. It's kind of like Aid Another, except you're interfering with the target to aid anyone they attack.
Being marked is rather distracting. This is where the Marked's attack penalty comes from. A combination of threat of harm and actual interference causes the Marked to either pay all due attention to the Marker or suffer the consequences. If the Marked decides to take their medicine and violate the mark anyways, the Marker is able to interfere with that attack because the Marked isn't paying full attention. If the Marked turns on the Marker and gives them full attention, then there aren't any openings to exploit.
The punishment mechanic (usually an Immediate attack) represents the Marker carrying through on the primary threat represented by the mark.
Non-Magical - Martial marks are the threat of physical attack. The marking character physically interfere with their target with feints, quick jab-like attacks and parries meant to foul the violating attack. If the target focuses on the Marker, he doesn't have the opportunity to make those distracting actions because the target isn't giving him openings.
Divine - Divine marks are the will of the gods. Paladins invoke their god's power to compel their target to honor their challenge and threaten them with smiting if they disobey. "You will fight me or taste the holy fire of Pelor!" The challenge itself is distracting and requires an act of will to disobey, which is represented by the attack penalty. If the target obeys the mark and takes the challenge, then the compulsion doesn't interfere and the gods trust their champion and the gifts they've already given them.
Arcane - Arcane marks are a diverse sort. They're cantrips of distraction and interference that the Arcane Defender can cast on a target they've prepped with a mark if given the slightest opening. They might be distracting illusions, prestidigitation, manipulation of time or shielding forces. They could be actively cast in combat or triggered contingencies keyed off of the marking effect. Arcane magic is wide open, so the exact form these marks take is limited only by your imagination and your character concept. The big gun (the punishment mechanic) is a more powerful spell effect.
Primal - I don't know what Primal marks would be like, I've never much been interested in that part of the game. Probably a combination of Divine and Arcane effects. Basically the same idea, though: the Marker threatens and interferes with the Marked if the Marked doesn't devote full attention to the Marker.
Regardless of the form the mark takes, the Marked is aware of what is going on and the choice it has to make. In the case of purely physical marks, this is pretty obvious: the fighter in your face with an axe is not a threat to be ignored lightly. Divine marks compel with divine power and must be actively resisted. Arcane marks are a little trickier, but probably rely on built-up magical power and overt spellcasting. Primal marks... I don't know, swarm the target with butterflies.
It might help to think of marking in terms of Opportunity Attacks. You get an opportunity attack if the target does something complex in your threat range, giving you an opening to sneak in a quick stab. A mark formalizes this and gives the Marked a choice: attack the Marker and prove you're focusing on them or declare that you're not focusing on them and give them an opening to do whatever they want to you. Violating the mark is the equivalent of saying "I'm not paying attention to you, I'm doing this other thing".