In "Dungeon Magazine #206", as a companion to the 4th edition adventure Eyes on the Ball, a lot of lore is added to the Xanathar guild. Here are some relevant excerpts
Over the years, many beholders have likewise claimed the title of Xanathar, some after killing their predecessors and some in a peaceful transition. The elder beholder Xandulzrithral fell into the latter category, choosing to retire from the role rather than continue to live as a target...
The penultimate Xanathar (once called Izulktur) held power for only a short time. It was originally part of a beholder cluster working alongside four others of its race... Eventually the creature grew bored... ordered its cluster to send a replacement.
...Zushaxx, a beholder new to the cluster, was interested in taking over as Xanathar. When Zushaxx took over...
Here are the inferences I can gather:
- If a beholder is killed by another to claim the title of Xanathar, obviously that beholder will immediately take over.
- If a beholder retires, they (or in the case of Izulktur, their cluster), will name a successor.
- If a beholder is slain by some other group, there may be a power vacuum for a short time, but another beholder will eventually claim the title.
In the first two cases, there is likely a relatively peaceful transition period. For example, in Izulktur's case, from the same tidbit in "Dungeon Magazine #206:"
Xanathar [Izulktur] ordered its cluster to send a replacement. The group sent a servant, a drow named Kal’dir, to Waterdeep to assist with the transition.
Kal’dir revealed that Zushaxx, a beholder new to the cluster, was interested in taking over as Xanathar. Before Zushaxx came to Waterdeep to assume the office, Izulktur took Kal’dir into its confidence and shared many secrets. The drow was instructed to assist Zushaxx during the transition and serve the new leader. The cluster agreed to repay Kal’dir’s loyalty by exacting revenge on those who killed the drow’s family.
When Zushaxx took over, it immediately replaced the guild’s top lieutenants and promoted new agents, starting with Kal’dir.
I couldn't find any explicit examples of what happens in the third case (like where a lich killed the second Xanathar), but I suspect the guild's high ranking officials would vie to promote beholders they are friendly with in order to retain their position in the guild, or to gain favor with the Xanathar.
It is unclear what happens or would happen during such a vacancy, so a DM is welcome to fill in these gaps with a catspaw/figurehead/patsy if that's the story they want to tell.