I was wondering what would happen if a Lich's phylactery is a piece of salt or sugar and it dissolved in the ocean. Would the ocean become the phylactery or would the Lich die?
Assuming the salt-based phylactory was allowed, then this should count as destroying the phylactory, and the lich would not be able to reform its body when it was killed.
When you smash something, the parts of it all still exist. Scientifically, when you burn something, the ashes still contain the same atoms of the thing. If a dissolved phylactory counted as still existing, then those scenarios should still count too, and it would be an odd outcome.
Would I even be able to make a piece of salt a phylactery?
It would be unusual, but not restricted by RAW. I guess it would be carved rock salt or similar. Here is some rock salt jewellery - it would not take much to add an interior space to that.
Even if I can do it, would it be too game-breaking?
If the goal is to have a character that is near impossible to be permanently killed, it is not that game-breaking at level 17, because there are many ways to come back by that point.
However given that you want to set the character up as a villain at the end and have the other PCs fight them, it seems a little cheap, and in my opinion bends the rules too much to count as fun for anyone involved but you.
You might be able to find some clever way to hide find or move the phylactory that allows the character to come back once or twice extra before being destroyed permanently. Look through some spell options, or talk to your DM about this. The goal should ideally be to make the betrayal fun and epic for you all . . . no need to look for rule bending ideas when you have the DM on side with the idea. Maybe get some powerful minions assigned etc.