Without retraining, you can do nothing special with a duplicate feat; unless the feature granting the feat says otherwise (perhaps by stating "If you already have this feat from another source..."), you merely have two copies of the feat.
And you're right that the feats' benefits don't stack. The PRD
has this to say on the subject:
...If a character has the same feat more than once, its benefits do not stack unless indicated otherwise in the description.
If this were 3.5, I might suggest embrace the dark chaos
then shun the dark chaos
spells (Fiendish Codex II
) but even that smacks of retraining, if you were intending to avoid it.