I emailed Kromm, here is his response.
It applies to both. See:
Size modifier does matter in melee.
"It is a modifier to rolls to hit you in combat and to Vision rolls."
(B19, Size Modifier).
The warrior would get both a -7 to hit the skull and a plus for the size modifier. If the dragon's longest dimension is 30 yards, then the warrior would get a +7 to hit, thus equalizing the penalties.
Of course, if the warrior is jumping (and thus moving) over the dragon's head, you must apply all the penalties to his attack first (in this case +0 as it evens out) and then reduce his skill to 9 or adjusted skill-4 (whichever is worse) for doing a move and attack.
Here is the word of Kromm (the Line Editor and Rules Arbiter of GURPS, also know as Sean Punch (the name on the front of all the books.)