The Armor of Bones is not a fragile armor
The supernatural ability creates a magical armor of bones, but not a mundane armor made out of the bone material (which is fragile). You will notice that several oracle's mysteries work similarly, and they emulate the effects of the Mage Armor spell (+4 armor bonus to AC for a few hours per day).
Since the ability (nor the spell) does not mention what material the armor is actually made out of, you can assume the material is irrelevant and no special rule should be applied to it.
Even if the GM insists on using the flavor of the ability against it, keep in mind that magical bone armor loses the fragile quality. Being a supernatural ability, this armor is naturally magical.
Masterwork and magical fragile weapons and armor lack these flaws unless otherwise noted in the item description or the special material description.