There's already an A+ answer.
Thomas Markov's answer is mechanically correct, especially for the GM.
…but, this might still be a bad idea…
That said: as both a player and a GM, I strongly advise caution in trying to bring insanity to your character - mechanically or otherwise - if the campaign doesn't already involve it. While it can be great fun for all involved, it is extraordinarily easy for it to become incredibly painful for everybody else: My Guy Syndrome is lurking in the wings, preparing to leap out and devour the campaign.
Let me expand on that a bit.
At a very basic level, this question is surprisingly similar to "how do I make my character a member of a spaceship crew?".
- For both questions, there are optional, GM-facing rules if the campaign calls for it (for the this question, as mentioned, the Sanity system; for spaceships, there's a good argument that the Spelljammer book's ships would work).
- For both questions, there are varying levels of GM buy-in available.
- For spaceships, the GM could potentially go with anything from "you survived the wreck of a spelljamming ship, but that's the last we'll hear of them" to "we're playing a Spelljammer campaign".
- For this question, there are options from "just be a little weird at times" to "we're all using the Sanity system".
- The question assumes a lot about the game world and campaign the GM has in mind - and many answers will force decisions that may or may not make sense within that context.
- For spaceships, the question implies that there are spaceships flying around the game universe, that some crewmembers are viable adventurers in this world, that this world is interesting to at least some spacefarers, etc..
- For sanity, well, that goes a little deeper.
You got Sanity in my Sword-and-Sorcery fantasy!
The core meta-game assumption of D&D is that there are several real humans sitting around a table, playing a game together. Those people have agreed to set aside a chunk of time - often a fairly sizable one - to cooperatively tell a heroic story about their characters. The rules seem to be written such that PCs succeed about 65% of the time they roll a d20 and they make it reasonably difficult to accidentally kill a PC: the PCs are the heroes of the story, and will very probably win in the end.
Stories about losing sanity often run counter to this assumption. My experience with Call of Cthulhu is that the story is more about finding the least-bad way of losing than it is about finding a way to win (which in one case involved calling a literal airstrike on our own position as a plan B). The stories were fun to play and re-tell, but they would have been a hard left turn had we come expecting anything more light-hearted.
Sword-and-sorcery?
At its core, D&D is designed around facilitating a sword-and-sorcery fantasy tale. Powerful heroes slay dragons and free princesses; wise wizards foil the plots of evil sorcerers; dashing rogues protect their fellows from fiendish traps and relieve evildoers of their ill-gotten gains.
Focusing on a descent into madness – on a PC's descent into madness - runs afoul of that assumption. That's not to say that it can't be done, but it's not what the system is designed for nor is it what most people are signing up for when they agree to join a campaign.
…that might hit too close to home…
Most portrayals of insanity or mental illnesses in fiction are, generously, "inspired by actual events". If anyone at your table has a mental health concern - or a friend, loved one, etc. who does - focusing on mental heath may very well cause discomfort, hurt feelings, broken friendships.
…and play out poorly.
My Guy Syndrome is the observed problem of a player justifying obviously-bad in-game choices because "that's what my guy would do".
If one player's character has a weak grasp on sanity, it seems reasonable to assume that their choices would be less rational to an outside observer. Thus, the player is more likely to make obviously-bad choices because "that's what my guy would do". This very often leads to broken campaigns and strained friendships: you've come together to play a cooperative game; one player choosing to screw up on purpose isn't fun for the rest of the players.
Note that, by "obviously-bad", I don't mean in a "least-bad" sense or with the benefit of hindsight, but in a "that's clearly a dumb idea" sense. "My Guy Syndrome" has brought us such hits as "I steal the king's crown from atop his head", "my guy will never attack the first round of combat", or even ... shall we just say that "seduction" stopped being the right word a long time ago. Critically, these are all things that the player chooses to do by themselves; if the rest of the players agree, it's almost certainly not My Guy Syndrome.
I could very easily see someone talking themselves into "insanity means randomly selecting a target each round, even from among the other PCs". This may be an extreme example, but it's the kind of trap that "my guy's less than fully sane" can lead to.
So, what do I do?
Talk to your GM and the rest of the players about your concept. If they're hesitant, it's probably best to drop it for this campaign.
If they rest of the players are interested, I would council working with the GM on an "escape hatch" in case something goes off the rails (this could include anything from the character exiting the campaign to a deus ex machina to resolve the insanity). I would also recommend looking more at Firefly/Serenity's River Tam or Psycho's Norman Bates for inspiration than, say, the cast of Alice in Wonderland.
That is to say:
If the GM and the other players don't want a space cadet in their party, well, that just isn't the hill to die on.
