My own solution is to add a caveat to the the help action:
If the creature with the highest ability modifier can not complete the task alone, then the help action becomes a "group effort", becoming a group check without advantage.
Each participating character performs a Help action, but at the end of the round a group check is performed to see if the task is performed. The GM should therefore adjust the DC according to who is participating and how many are participating.
The rules concerning Group actions can be found in PHB p175
When a number of individuals are trying to accomplish something as a group, the DM might ask for a group ability check. In such a
situation, the characters who are skilled at a particular task help
cover those who aren't.
To make a group ability check, everyone in the group makes the ability check. If at least half the group succeeds, the whole group
succeeds. Otherwise, the group fails.
The DM should determine the difficulty check depending on type of challenge and number of people trying to do it. For example lifting the same boulder might be DC20 if 4 people attempt it, but DC 10 if 10 people try to lift. While scrambling up a wall in a timely manner might be a flat DC15 regardless of number of people, with the stronger/luckier ones helping the weaker/unluckier ones.
- A group check would emphasize the task is a team effort.
- No advantage is given to distinguish the it from a regular Help action. This makes it easier to understand and removes confusion.
- The task is performed at the end of the round so as to allow other players to join in, while also allowing some enemies to try and stop the plan. This is done for simplicity sake, as an arbitrary action or reaction would be too convoluted. This can be explained in-game as the time required to coordinate the task.
Why the roll?
Other answers mention pooling together modifiers which would then succeed a threshold. While effective, it isn't as tactical. If enough characters contribute, then the plan always succeeds. If it is an attack, however, the enemy can make saving throw.
This is where instead the aforementioned group check comes into play. Players must judge the risk of contributing . Perhaps too many characters contribute and they overcompensate, wasting actions that could have been spent more wisely. Or, perhaps a plan fails and a player regrets not contributing to the effort.
This method allows for failure in combat for group efforts that are not an attack, for example: scaling up a chimney. This stands in contrast with the threshold method which does not afford failure in such situations.
When is it a "Group Effort"?
If the task can possibly be performed by a participating member, then it is regular Help assist. Else, if no participating member can perform the task, it becomes a "Group Effort"
A Goliath and a Dwarf want to ram a spear through a Dragon's heart. The GM ask themselves whether the Giant or the Dwarf would have enough strength to do it alone. If at least one of them can feasibly do it alone, then it becomes a regular Help assist. Else, if both our incapable of doing it alone, then it becomes a "Group Effort".
The GM states that the task would be group effort. The dwarf moves, reacts and takes the Help action on its turn. The Goliath does the same. On the end of the round a group strength check is made against a DC of say 20. If an hulking orc joins in the effort, the DC is reduced to say 10. If a tiny gnome joins in, then the DC is reduced to say 15. If both the orc and the gnome join in the effort the DC is reduced even further so say 5.
Finally a group strength check is performed. If it passes, the spear manages to pierce the dragons heart. If the group fails to pass, then the spear bounces of the Dragon's scales.
Three mages want to send glass shards hurtling at their enemies. One might conjure sand, the other fire and the last one air. Since the action is impossible to perform alone, this becomes a group effort.
Each mage uses it's action to Help. At the end of the round, they pass a group intelligence check. If they pass, then they skewer their enemies with a thousand glasses pieces. If they fail to pass the intelligence check, then the contributing magic spells are misaligned.