First, kudos for a fascinating question.
Game is DnD 3.5, not that it matters.
Actually, it might.
Another responder linked a newbie-friendly roleplaying question. It's possible that the mechanics of a game may get in the way of a player's expectations or such. I can see how the mechanics of, say, D&D 4.0 get in the way of my having fun, so perhaps it's possible for something similar for 3.5 and her.
... and when I asked Rogue what she wanted to do, she said "I go to sleep". I was like "Okay, you go to sleep" and then she literally went to sleep. LARPing at it's finest.
There might be two ways to approach this. One is for the other players to not be so independent and to talk as a group and do things together. It sounds like they are quite experienced. The other is for you to ask (something like) "are you sure?" or make small suggestions to inspire her. Maybe she just doesn't know what to do. It does feel like she's not really in her character's head.
I'd hope that the other players pull her in with their plans more often and for them to feel just as responsible for one another being engaged as you do. It's really not all on you.
And I joked about trying not to award those who fall asleep during play. She suddenly got sensitive about my joke.
You were mean. Pulling her to the side on a non-gaming-day and talking with her may have been a better way to communicate what you feel is her lack of participation.
I've done xp based on participation, where every player really had to fight for their xp. It was fantastic because it engaged us all, and although we never knew what other people got for xp it did give us a superiority-thrill. I definitely wouldn't recommend this in an environment where some players are less active than others.
Even when she plays, she is pretty much disinterested all the time. During combat, when she rolls one or two bad attack rolls, she is like "Well, I can't hit anyone anyway" and it's like we almost force her to roll the die.
This is part of the hassle of roleplaying for newbies.
I know some people who are lucky all the time, and some who are lucky in specific circumstances. It's odd. For what I've seen, what really catches people's attention and makes them pay attention to their rolls is the fight between predicting what they're going to roll based on trends and what they actually roll. I know some players who are amazing with hit rolls but crap with damage rolls. We joke about that all the time.
Does she own her own dice? If not, hand her a (new!) set as a gift. There's a perception of a personal bond between a player and their dice. I don't know how to explain it, but I've seen this help players work out their issues with luck.
Then she starts playing with her phone while others make their move.
This is difficult to say no to, but it does have to stop. I don't think I'd recommend banning it, but this would definitely be a useful sign to see how engaged she is at any particular moment.
That's pretty much it. She still cares to come to the sessions at least.
I think that's a positive sign.
I'm setting up a quest about her character's background. If that doesn't work, I don't know what else I can do.
I don't feel that would work. It feels like there's some undercurrent which needs to be addressed. Try it though.
I don't want to kick her out of the group but things are going in that way. It just downs everyone else in the group and kills the mood.
In my elitest days, the environment I ran would have had her either quietly drop out or ragequit. I don't like the old me. Her bringing the other players down is horrible, and the strongest reason to remove (or improve) her. Still, she deserves a chance and a chat.
So, how do engage an uninterested player? If all fails, how do you deal with her?
"deal with her" is probably your biggest hurdle.
Maybe spin the problem as "I have a player who isn't engaged like the others, what can I do?"
I also want to mention how deep-rooted my bias is regarding female roleplayers. I'm quite aware of it, and working on it, and this question made me struggle a great deal. Others asked about her relationship status to others in the group. I didn't even realize that was in the back of my head before reading those comments. Hopefully my spinning the problem at the end here shows how I perceived it.