In The Castle Guide supplement page 54 is a table detailing several building types and their associated costs and requirements. Are the values under the column for build time meant to represent person-weeks or person-days?
Person-Weeks, or "man/weeks" as the guide calls it.
These numbers are later given units as the guide explains how to put everything together. We see in section 5.7.4:
Eight small, square tower modules cost a total of 112,000 gold pieces to build and require 6,720 man/weeks of work.
From the table, one square tower module is given 840 units of time, so 8 of them would be 6720, which the above quote defines as man/weeks.