Without retraining, you can do nothing special with a duplicate feat; unless the feature granting the feat says otherwise (perhaps by stating "If you already have this feat from another source..."), you merely have two copies of the feat.
And you're right that the feats' benefits don't stack. The
PRD has this to say on the subject:
...If a character has the same feat more than once, its benefits do not stack unless indicated otherwise in the description.
If this were 3.5, I might suggest
embrace the dark chaos then
shun the dark chaos spells (
Fiendish Codex II) but even that smacks of retraining, if you were intending to avoid it.