I'd look at something like this as imposing Disadvantage. It's distinctly harder to hit a specific target when they are closely interacting with someone/something you don't want to hit. You don't need a lot of clearance to do so when the target and the thing you want to not hit are stationary, but that's not going to be the case when two (or more) people are engaged in combat.
No, it's not explicitly listed as being something that imposes disadvantage, but it does seem to fit the description on p7.
Sometimes an ability check, attack roll, or saving throw is modified
by special situations called advantage and disadvantage. Advantage
reflects the positive circumstances surrounding a d20 roll, while
disadvantage reflects the opposite.
The 'Advantage and Disadvantage' section on p173 starts with similar text, but at the bottom of the section includes the following:
You usually gain advantage or disadvantage through the use of special
abilities, actions, or spells. Inspiration (see chapter 4) can also
give a character advantage on checks related to the character's
personality, ideals, or bonds. The DM can also decide that
circumstances influence a roll in one direction or another and grant
advantage or impose disadvantage as a result.
If your party is fighting a much larger creature (e.g.: a dragon or giant), and it's between you and your ally, it may be easy enough to avoid hitting your friend that you aren't at a disadvantage. If you're fighting something small and mobile (e.g.: a pixie or stirge), on the other hand, it may be quite hard. This would, properly, be a DM's call. Just try to be consistent about the logic you use to determine when to apply it or not.