It depends on the nature of the Help.
The Help action does two discrete things.
Ability Checks
You can lend your aid to another creature in the completion of a task. When you take the Help action, the creature you aid gains advantage on the next ability check it makes to perform the task you are helping with, provided that it makes the check before the start of your next turn.
This is the more variable of the two. If one character is helping another pick a lock (by holding a light source, providing tools like a nurse to a surgeon, etc), the helping character needs to be close to the acting character. If one character is helping another climb a wall by hauling on a rope, they're connected by the rope but the distance could be far greater.
Attack Rolls
The key word for considering these cases separately is "alternatively". They're two paragraphs, and the second leads with a dividing word, not a joining word.
Alternatively, you can aid a friendly creature in attacking a creature within 5 feet of you. You feint, distract the target, or in some other way team up to make your ally's attack more effective. If your ally attacks the target before your next turn, the first attack roll is made with advantage.
The victim of the helper (the second bold section) has to be within 5 feet ("you" is the helper). There is no matching location requirement for the character being helped (the first bold section).
Of course, there's nothing saying the helper has to stick around after taking the Help action. The Sage Advice Compendium confirms it:
If you use the Help action to distract a foe, do you have to stay within 5 feet of it for the action to work?
No, you can take the action and then move away. The action itself is what grants advantage to your ally, not your staying next to the foe.
Some characters, like Mastermind Rogues (SCAG, p. 145; XGtE, p. 46) don't even need to be that close - they can do it as a bonus action from 30 feet away from the victim.