Modifiers for ability checks are with respect to each check.
Ability Checks
To make an ability check, roll a d20 and add the relevant ability modifier. As with other d20 rolls, apply bonuses and penalties, and compare the total to the DC.
Advantage and Disadvantage
Sometimes [...] you have advantage or disadvantage on an ability check [...]
You usually gain advantage or disadvantage through the use of special abilities, actions, or spells. Inspiration can also give a character advantage. The DM can also decide that circumstances influence a roll in one direction or the other and grant advantage or impose disadvantage as a result.
If multiple situations affect a roll and each one grants advantage or imposes disadvantage on it, you don't roll more than one additional d20. If two favorable situations grant advantage, for example, you still roll only one additional d20.
If circumstances cause a roll to have both advantage and disadvantage, you are considered to have neither of them, and you roll one d20. This is true even if multiple circumstances impose disadvantage and only one grants advantage or vice versa. In such a situation, you have neither advantage nor disadvantage.
Special abilities, actions, spells and Inspiration are in the player’s domain. Some circumstances are called out in the rules, and some are ad-hoc, adjudicated by the DM.
A bonus or penalty applies to the ability check.
Contests are different, but modifiers still apply to each check.
Both participants in a contest make ability checks appropriate to their efforts. They apply all appropriate bonuses and penalties, but instead of comparing the total to a DC, they compare the totals of their two checks.
A penalty on one side, does not cancel out a bonus on the other side.
When we run skill contests, especially Stealth contested by Perception, we sometimes struggle to work out whether to apply advantage to one skill or disadvantage to the other.
For the modifiers defined by the rules, this is determined for you.
For ad-hoc adjustments, narratively, you should apply the modifier to the side that is the source of the modifier. If you determine an effect impacts the ability of one actor to perform a task, that should be advantage or disadvantage on that actor's ability check.
For example, if a rogue is trying to sneak passed talking guards, you might grant disadvantage to the guards' Wisdom (Perception) checks, since they are distracted by their conversation. However, if a cat pounces in front of the rogue from atop a wardrobe, you might impose disadvantage on the rogue's Dexterity (Stealth) check, as it might distract the rogue. If one of the guards leaves, you might grant advantage to the remaining guard, as she returns to vigilance.
For simplicity, let’s assume in your example that all enemies are aware of each other, the Order of Combat has been followed and this occurs during Step 4, Take turns. Since the cultists are carrying lights, let’s assume they do not have darkvision and the lights are torches, and the fighter has darkvision.
Other Adventuring Gear
Torch. A torch burns for 1 hour, providing bright light in a 20-foot radius and dim light for an additional 20 feet. If you make a melee attack with a burning torch and hit, it deals 1 fire damage.
- The fighter's Stealth check had advantage owing to being heavily obscured in the darkness
The fighter does not need to make a Stealth check to hide in a heavily obscured area.
The Environment
A heavily obscured area--such as darkness, opaque fog, or dense foliage--blocks vision entirely. A creature effectively suffers from the blinded condition when trying to see something in that area.
- The fighter's Stealth check had advantage due to her boots of elvenkind
- The fighter's Stealth check had disadvantage due to her armour
Neither is relevant since no check is required.
- The cultists' Perception check had advantage because they had already been attacked by the fighter,
The cultists have no chance of seeing the fighter in a heavily obscured area. If on the fighter’s turn in initiative order, the cultists have ended their move within 60’, but outside of 40’, the fighter would get one round of being in a heavily obscured area, effectively invisible to the cultists, at which point the fighter could make an attack with advantage, with no ability checks being relevant.
However, once the cultists move within 40’ with their torches, the fighter would now be in a lightly obscured area (for the cultists). The fighter would need to take the Hide action, and would not be able to use the Attack action until the following round unless the fighter had some feature like a Rogue’s Cunning Action. Ideally, if there were previous rounds in which the fighter did not attack, the fighter would have already taken the Hide action. The cultists’ Perception checks are Passive, and have disadvantage.
If on the cultists’ turn, they move within 20’ of the fighter, then no checks are necessary as the fighter is now in bright light. The fighter could play cat and mouse, moving into darkness on each turn, but without a Rogue’s Cunning Action or similar ability, the fighter would not be able to attack, and the cultists could simply move within 20’ and attack normally.
Cunning Action
Starting at 2nd level, your quick thinking and agility allow you to move and act quickly. You can take a bonus action on each of your turns in combat. This action can be used only to take the Dash, Disengage, or Hide action.
Stealth
To determine whether such a creature notices you, the DM compares your Dexterity (Stealth) check with that creature's passive Wisdom (Perception) score, which equals 10 + the creature's Wisdom modifier, as well as any other bonuses or penalties. If the creature has advantage, add 5. For disadvantage, subtract 5.
Vision and Light
In a lightly obscured area, such as dim light, patchy fog, or moderate foliage, creatures have disadvantage on Wisdom (Perception) checks that rely on sight.
Bright light lets most creatures see normally.
In a lightly obscured area, the fighter’s check is normal, since that ability check has at least one source of advantage and at least one source of disadvantage. The rules-consequences of the fighter attacking the cultists is that the enemy would know the fighter’s location, however, that was prior to the fighter moving into darkness and does not apply until after the fighter attacks from darkness.
Unseen Attackers and Targets
When you attack a target that you can't see, you have disadvantage on the attack roll. This is true whether you're guessing the target's location or you're targeting a creature you can hear but not see. If the target isn't in the location you targeted, you automatically miss, but the DM typically just says that the attack missed, not whether you guessed the target's location correctly.
When a creature can't see you, you have advantage on attack rolls against it. If you are hidden--both unseen and unheard--when you make an attack, you give away your location when the attack hits or misses.
- The cultists would now have made a standard Perception check because their existing advantage would have been cancelled out by the new source of disadvantage
- The net result would then have been: a standard Stealth check was contested by a standard Perception check.
I'd given the cultists an unwarranted leg-up.
The sources of advantage and disadvantage on a contested check are with respect to each check and should not have any bearing on the opposing check.
Bottom Line
When a bonus applies to one side, and a penalty applies to the other side, you might be tempted to cancel them out, but it would be a house rule that may significantly change the probability of the outcomes. Depending on the target DC, advantage or disadvantage can range from +/- 1 to 5, and be different for each opponent, depending on special abilities, ability scores and proficiency.
Further, the Lucky feat or a halfling's luck don’t get the commensurate benefit or detriment if you simply cancel out the sources of advantage or disadvantage on both sides.