When refereeing a game, the following question came up: Do celestials weigh the same as their base template creatures? For example, if a large celestial is injured and needs to be carried out of battle, how do I decide if "it can be done"? Or, if a large celestial is crossing an unstable bridge, how do I adjudicate if the bridge will collapse? Presumably, a planar creature is subject to the laws of the plane it inhabits, but celestial template features include resistance to magics which imply that the laws of "physics" affect them differently.
There are no rules that say celestial templated creatures weigh more or less than the base creature. In particular:
In fact, all of their features are supernatural: elemental resistance, spell resistance, damage reduction, darkvision and smite evil. Magical abilities tend not to "weigh" anything.
That said, it's your campaign, and descriptive elements like creature weight are yours to change as you like. The rules merely don't suggest a change in weight, but you're absolutely free to decide that certain extraplanar beings are made of different stuff and weigh more or less than a normal creature.
Resistance to magic implies nothing about the laws of physics. In fact, every spells that create something real like a hail of stone or a acid globe is not influenced by spell resistance.
There's no reason why the celestial creatures should weight more or less than their counterparts, not in the rules at least.
You could decide celestials weight less or more according to their specific ancestor, the presence of wings or such other elements, like it's done for genasi, but I've never seen that being considered in RAW.