At what point do you leave a creature's reach when moving vertically?

Question

The scenario: creature 1 (C1) is flying (and can hover) directly above creature 2 (C2). They are in melee combat with each other and both have 5' reach. They are both medium size, both are 6' tall, and the melee is being run on a grid.

How far, vertically, above C2 does C1 have to be to be outside their controlled space and how far from C2 does C1 have to move vertically upward before provoking an opportunity attack by attempting to leave their reach?

What is the situation if C2 is also flying and attempts to leave C1's reach by moving vertically downward?

Please note that I chose 6' tall specifically as this means that the creatures are taller than the 5' height of a "grid cube". To accentuate this point add in a third creature standing next to C2 who is still medium sized but is 8' tall, i.e. a different height to C2 but still on the same grid and still the same size.

Also it is clear that an easy way to answer this would be to just enforce an idea that a medium creature occupies a 5' cube, but this goes beyond the "d&d is not a simulation" defence (which is true in most situations) as if the creature is over 5' tall then it clearly occupies a bigger space, let alone "controls" it. If the creature is an 8' hobgoblin, for example, then it's head, arms and shoulders are clearly in a different cube. The rule about control works for when the space is larger than the creature, but what about the other way around? I don't know any other examples of this.

Another example of where this is important is how high does someone need to jump to jump over a creature, both at all (avoiding moving through an enemy's space) and also avoiding OA (by leaving their reach). The rules calculate how high a creature can jump down to the foot.

The reason I am asking this question is because I am going to be running a game in a short while where there is going to be a Monk PC who is medium sized but 8' tall who is likely going to be protecting, and standing beside, another PC who is medium sized and 5' tall and they are going to be fighting enemies that can jump and others that can fly. When I was considering how to run a fight against either of these enemies it occurred to me that using a 5' cube 3d lattice just didn't make enough sense to me to qualify for the "its not a simulation" defence of how D&D works, that the 5' square grid rules didn't extrapolate well to this situation and neither I nor my players will be easy with the poor approximation, it is too far from that dreaded word "realism" to feel fair. Thus I either want RAW about how to handle the vertical and that is that, or I am going to use the variant rule only for horizontal considerations and do approx 3d Pythagoras in my head for the vertical distances. I'd prefer not to have the maths overhead to my job as DM.

Answers supported by referenced RAW or other official material please, not opinion about whether the 5' grid square rule just maps to a 5' cube 3d grid, because it clearly does not do so easily.

Answer 1

Moving one square vertically will trigger an opportunity attack.

The relevant general rule is simply:

You can make an opportunity attack when a hostile creature that you can see moves out of your reach.

(PHB, p. 195, "Opportunity Attacks").

And the rule for figuring ranges on a grid, which includes reach, is:

To determine the range on a grid between two things ... start counting squares from a square adjacent to one of them and stop counting in the square of the other one.

(PHB, p. 192, "Variant: Playing on a Grid")

D&D is an exceptions-based system, meaning that general rules apply unless there are more specific rules in place, and there's nothing in the single paragraph of rules for flying in combat (PHB p. 191) that change how these rules would operate.

So, if C1 can make melee attacks against C2, they are within one square of each other -- that is, they are adjacent on the grid. If C1 moves so they are more than one square away from C2, C2 can make an opportunity attack (and vice-versa). On a grid, all movement is in increments of one square, so any move from either combatant that puts more than one square of distance (making them no longer adjacent) between them is sufficient.

How big is a vertical square? The rules just say that a square "represents 5 feet". There are no specific rules for how to use a grid for three-dimensional combat (and remember that the rules for a 2D grid are themselves both optional and lightweight). The most direct interpretation of the rules is that a square on the grid represents a 5' cube of space. Thus while an active, flying, Medium creature will, strictly speaking, have bits of them sticking outside of the boundaries of "their" cube, what really matters is that any creatures who occupy any of the 26 adjacent cubes are within their one-square (or one-cube) reach.