Work with your DM, and prepare for disappointment
You'll have to talk to your DM to see what is and is not permissable. They would have final word on whether or not the spell is triggered.
Simple things like, "When I'm engulfed under water, trigger water breathing." is pretty simple. So is, "If I'm reduced to 0 hp, automatically cast a 5th-level cure wounds on myself."
But triggers of "something is near (or within 125 ft)", is a little more subjective. First, you'll have to talk about how far away the spell can sense a trigger. 125 feet shouldn't be too bad. Given that, it can be triggered by something in the next room, or even a few rooms away. Or the floor above or below you. And so on. So it could go off far more often than you suspect.
Next, what is "affect a creature's soul?"
The rogue's dagger can affect a creature's soul by sending it to the dark waters. A priest giving a blessing (not the spell, but ceremonial rites) could affect a creature's soul for the better. You don't want the spell going off because someone was courteous when you sneezed.
And last, and I've always wondered about this, is, where are you going to teleport to?
I see a lot of people talk about using dimension door with contingency, but I never see an explanation of where they plan to go. If you say, "away", in what direction is "away"?
If you would arrive in a place already occupied by an object or a creature, you and any creature traveling with you each take 4d6 force damage, and the spell fails to teleport you.
I would say that a good percentage of the time "away" will put you into a solid object: wall, another creature, side of a mountain, etc. Or maybe even a worse situation. Yes, you're farther away from the soul-sucker, but now you're in a room filled with monsters.
So to sum up... It may work, you'll need DM buy in, and hope you're DM isn't a jerk and either sets it off at inopportune moments or moves you from a neutral situation to a bad situation.