Within the official D&D 3.5e and Pathfinder 1e publication lines, I am rather confident, there are not any such rulesfor creating or improving special materials: the special materials are just offered, as-is, and you’re expected to just buy them (or pay the crafter extra for their value). There may be flavorful descriptions of the process by which special materials are refined, but this isn’t to improve the special material, it’s to get the special material in the first place. There also may be a few special materials that are described as “this other special material, but extra refined.” But what there isn’t is any kind of rules system for doing that kind of refinement.
Source: massive experience with these two systems.
I cannot comment on third-party products or other potential d20 System™ products that might be compatible; I have no knowledge of these.