These are, yes, the canonical size categories used throughout Pathfinder. It's plain when looking at them in context: every table on page 187 has the column headers Large, Huge, Colossal, and Gargantuan. Those tables look a lot like they're using Size technical terms because they are. You'll notice the same canonical size categories used on pages 159, 161, and 165 that describe siege engines.
But what about castles? Well, first off a castle isn't a building — a castle is a collection of buildings, walls, and other defensive structures (which under these rules are abstracted as individual buildings and as either walls, gates, or individual siege engines). Second, note especially that the 64-foot height definition is the lower bound of what the Gargantuan size covers: Gargantuan covers 64 feet and up / 125 tons and up. If you have a very big single building that should be targetable as a single building instead of a collection of walls (which isn't always sensible; see below), then it's either just on the larger side of Gargantuan or take this last line of the Buildings header (p. 167):
For larger buildings, put together multiple buildings of these sizes and add the hit points together.
But, keep in mind that very large buildings that are designed to prevent damage in some areas from collapsing the whole structure, well, shouldn't. If you want to abstract it away and have even enormous buildings have only "fine", "half wrecked", and "gone" statuses, then abstract it away as a multiple buildings with combined hit points. But if you want a finer understanding of the battlefield, a building so big that it is structurally equivalent to independent walls could be treated as Walls (Table 3-19) or even multiple independent Buildings instead of as a singular Gargantuan Building (Table 3-18).
Do these options mean that you might, as GM, have to make a judgement call as to how to treat very large buildings under these rules? Yeah, it does; that's why there's a human in the GM chair instead of a computer. Besides, you put the building there, so you should already know which abstraction matches the structural nature you imagined it having.
So then, why are there even Large buildings listed? Well, wouldn't it be annoying if your PCs aimed a catapult at the witch's hut, or the mayor's fancy brick outhouse, and Ultimate Combat didn't have stats for it? Instead, it has you covered with Large.
- Aside: beware that the Buildings heading on page 167 contains text that swaps Colossal and Gargantuan. You'll see what I mean when you look at the prices for “magically treating” buildings, but it's also evident in the notes about their ACs, since it should probably read “Colossal or larger” where it says “Gargantuan or larger”, since otherwise they skipped the AC for Colossal buildings. The writers and editors of that passage seem to have written from memory but misremembered which of Colossal and Gargantuan is the bigger size.