I'm creating some maps with a 5ft hex grid and hoping to keep them system agnostic. There are some rooms that have a shape that doesn't fit the hex grid well and would fit a square grid much better. Is there any problem with transitioning between the grid types if the transition is clear and simple?
I frequently do this
As PipperChip points out there are some issues that can theoretically arise, particularly where the two grid systems intersect.
I use a hybrid system heavily when curved stairwells exist. However, in these cases, I actually use a third system: radial grid (think dartboard). I'm not saying this is a perfect system or that there isn't better way to do it. But I do think it generally makes better use of curvilinear spaces than a hex grid does.
I'm running Curse of Strahd for a group right now and there are a number of long curving stairwells around the outside of round rooms. In these situations, I build a custom map that uses a hex grid for the center of the tower and a radial system for the stairwells. Where the two meet, the lines can get a little unusual looking but, from a movement and size perspective, they are pretty clean and intuitive to use.
Below is an example of going from Hex to Radial. It does create deadspace pockets along the wall in places but I often let characters occupy half spaces. Sometimes I'll tell players that they have to squeeze to fit into the space if it's a particularly truncated hex.
Another thing I do when using a hex grid system in a square room is to reshape the peripheral spaces along the wall to be a hex on one side and a square on the other. They might technically be a little different size/area than the rest but this is infinitely preferable to having half a hex be the space right against the wall and figuring out if a creature can occupy it.
Here's an example of when I switch from hex to square. In this particular map, the square grid suddenly ends (you can just see it at the top of the image). Our group has a lot of Dwarven Forge set pieces which use a square grid and we used them to make a wall (think battle of Helm's Deep from the Lord of the Rings). This map was printed out and tucked up against the set pieces to act as a transition from a big open field that used the hex grid to the square grid of the wall pieces.
I suppose it is possible to find a way to exploit the interface of these two systems when it comes to AoE or creature size but what I've found is that this hasn't been a problem for me.
For one, a significant proportion of AoE spells say that they wrap around barriers or simply ignore cover so you can often just measure the radius directly instead of counting spaces.
Second, the simplification for positioning and movement in the stairwells and doorways vastly outweighs any added complexity or balancing issues that might arise for AoE effects and positioning. I do have the benefit of playing with some easy going and understanding players, though, so your mileage may vary.
I think that you can help further mitigate issues by 1) explaining how you intend to handle any issues relating to size/position and AoE that might arise ahead of time and 2) generally ruling in favor of the players when they do.
Only if your players are OK with it.
There are some issues involving AoE and creature sizes effects:
- Any effect which hits one space and the surrounding spaces hits 9 spaces on a square grid but only 7 on a hex grid.
- AoE which originate at a corner and expand outward still don't cover the same amount: 1 space from a 'corner' is 4 spaces on squares and 3 spaces on hex.
- A similar thing goes for creature sizes! How big is a more-than-one-space creature? 3 spaces on hex but 4 on square?
These differences are usually 1 or 2 spaces and don't become huge until you go big!
Additionally, squares are go further/faster when you go along a diagonal. This is why some systems have some sort of rule to compensate for diagonal movement (I'm looking at you, D&D 3.5!)
In any case, the differences are not negligible and your players should at least agree to it before you spring it on them.