As the DM, when one of my players is being boring or distracting, I usually just interrupt and remind everyone about the plot. I'll turn to someone who's not currently engaged and I'll say: "Okay, so you've showed up at the museum, and it's pitch black inside, like someone's using darkness magic. Somewhere inside the building you hear a little boy giggling. There's quite a lot of blood outside the museum, but you don't see any corpses. What do you do?"
(I do this sort of recap frequently, even when nobody's being distracting. I feel that it helps to set the scene and keep the game moving, and also I really enjoy giving these two-sentence dramatic plot summaries.)
Most players take this pretty well -- they understand that it's the DM's job to keep the game moving. Ideally, when I remind my group of the problem they're supposed to be solving, it even gets the annoying player to focus on the problem instead of whatever side path he's focusing on.
It's also worth noting: it sounds like you have six players, which is really too many -- most groups are best with four, for pretty much the reasons you're describing. (When there are six players, each player only gets to talk one-sixth of the time, which is not enough.) It's probably not necessary to ask your problem player to leave, but if you did have to drop to five players instead of six, it would leave you with a better game.
You've noted in a comment that this problem "happens specifically when the party splits up (shopping days, separate inn rooms, etc.)". This leads me to another suggestion: don't let them do that. :) Much of the fun of a D&D game is in doing plot-related things as a group: exploring dungeons, fighting monsters, investigating eldritch magics. Very few players are playing D&D so they can roleplay going shopping, and basically nobody is playing D&D so they can listen to other people roleplay going shopping.
What I do for shopping and downtime is I ask my players to do it outside of the session, by emailing me (or emailing the group). This might work especially well for you: you can end a session in town and ask everyone to email the group with what they're doing during downtime, and if someone wants to send a two-page email describing what color his eyes turn as he drinks tea, that's up to him. Game time, when everyone is present, is used for advancing the story.
If the group gets separate inn rooms, I handle it like this:
A: "Okay, so we're getting rooms at the Yellow Dragon Inn. I'll get my own room, paying the extra three silvers."
B: "I'll room with C, and we can spend some time talking about my favorite topic: swords! I like shortswords and katanas and broadswords and scimitars and --"
DM: All right, you guys get rooms and sleep the night. Everyone take a long rest.
DM: The next morning you wake up and Farmer Macdonald is pounding on your door. He says two more of his chickens went missing overnight, and there's this horrible green thing where one of the chickens was nesting, and it's got at least five eyes and it gives him a horrible headache when he looks at it. "What's happening?"
DM: What are you going to do?