My question is: How can you use the animal handling skill in a mount? does it help to controll it, to make it not be afraid of something, etc or is just DM-dependent?
In 5e, all ability checks, including those which use Animal Handling as a Proficiency, are called for at the DM's discretion. There's many examples of checks which are reified into actions the PCs can take―for example, the Strength (Athletics) check made when attempting to Grapple an enemy combatant, who responds with either Strength (Athletics) or Dexterity (Acrobatics)―but I'm not aware of any specific actions PCs can take that unambiguously force a Wisdom (Animal Handling) check. So unless a specific spell or class feature says so, Wisdom (Animal Handling) checks are made at the DM's discretion.
What the Player's Handbook suggests:
When there is any question whether you can calm down a domesticated animal, keep a mount from getting spooked, or intuit an animal's intentions, the DM might call for a Wisdom (Animal Handling) check. You also make a Wisdom (Animal Handling) check to control your mount when you attempt a risky maneuver.
―Animal Handling, Player's Handbook, pg. 174
As DM, these scenarios are mostly consistent with how I use Animal Handling at my table, although in the specific case where you're trying to calm down an animal, I'd probably use Charisma (Animal Handling) instead of Wisdom, because Wisdom as an ability score more keys to intuition and empathy, whereas Charisma more keys to communication.
Other ways that I use Animal Handling at my table
The first thing I'll say is that, in most situations, I don't call for checks at all. This is because all of the mounts that my party (currently) has access to are trained beasts of burden that were rather expensive to purchase, and were specifically trained to be easy to use in non-combat situations.
So the major exceptions are when the party tries to use their mounts in combat, or otherwise in hostile territory. This is a non-exhaustive list of scenarios I've called for Ability checks using Animal Handling at my table:
- Giving their mount a name and trying to determine if the mount likes their name or not called for a Wisdom (Animal Handling) check
- If the animal is actively startled and trying to buck its rider, I'll usually call for a Strength (Animal Handling) or Dexterity (Animal Handling) check to determine if the rider can stay mounted. Which ability score I use is mostly just a matter of discretion regarding what the character is good at, though the Strength checks usually have lower DCs than the Dexterity checks in this case
- Attempts to assess and analyze the personality/physicality of a mount (usually at the time of purchase) have called for Intelligence (Animal Handling) on a few occasions
- A poorly trained mount has required periodic Wisdom (Animal Handling) checks for the rider to figure out how to direct and guide it
Like I said, this is non-exhaustive, it just gives a general sense of the kind of checks I've called for to deal with interactions with mounts. There's probably also a whole host of other scenarios that don't deal with mounts specifically that would also call for these kinds of checks, that just don't happen very often in my campaign specifically.
The skill description of Animal Handling offers some basic guidance for when a DM might call for an Wisdom (Animal Handling) check:
When there is any question whether you can calm down a domesticated animal, keep a mount from getting spooked, or intuit an animal’s intentions, the DM might call for a Wisdom (Animal Handling) check. You also make a Wisdom (Animal Handling) check to control your mount when you attempt a risky maneuver.
Beyond this, we do have some examples from published adventure's that give us some more details about how we might use the Animal Handling skill.
Racing and Chasing
The adventure module Tomb of Annihilation has a rather extensive dinosaur racing system built around the Animal Handling skill:
A race runs a length of 300 feet; this is abstract, as a race actually covers a lot more ground. Every round, each rider makes a Wisdom (Animal Handling) check; the DCs for different types of dinosaurs are listed in the Racing Dinosaurs table. With each successful Animal Handling check, the first number listed as the dinosaur’s speed is added to its “running tally.” If the check fails, that dinosaur’s tally doesn’t increase that turn. When a racer’s tally equals or exceeds 300, that dinosaur crosses the finish line. A racer can try to move at the higher listed speed by lashing the animal furiously; in this case, the Animal Handling check is made with advantage, but the dinosaur must also make a successful DC 10 Constitution check at the end of this round or its speed is halved for the rest of the race.
This gives an idea of how you might use Animal Handling while racing on your mount, or while chasing or being chased; really any situation where you're trying to move fast and want to add some dice rolling to decide outcomes.
In Explorer's Guide to Wildemount, we see these situations in one of the short modules contained therein:
A character who succeeds on a DC 18 Wisdom (Animal Handling) check is able to calm all the dogs and prevent them from barking or attacking.
If Bol’bara has already been possessed, the dogs survive her initial onslaught but are fearful in the aftermath. A character who succeeds on a DC 14 Wisdom (Animal Handling or Insight) check discerns that the dogs are too frightened to attack. If the dogs are freed, they either flee or can be encouraged to come with the characters at your discretion. If they aren’t set free, they die of thirst and starvation within a few days.
If your mount is frightened by something, an Animal Handling check would be used to determine if you are able to calm it down.
Meeting New Companions
We also see examples of Animal Handling being used to meet new companions, in Infernal Machine Rebuild:
If the bears are freed, they scatter throughout the temple, attacking anyone they see before eventually making their way to area 1 and up the stairs. If Barbatos is still in the room when the bears escape, they attack him first.
A character who succeeds on a DC 20 Wisdom (Animal Handling) check can convince at least one of the bears (of your choice) to remain with the party for the duration of the adventure. Feeding the bears first provides advantage on this check.
So here we see an Animal Handling check being used to convince a recently freed bear to stick with the party and help out - this can similarly be used by the DM to find and connect with new mounts.
A DM may consider using a different Attribute
Wisdom is the canonical attribute used for Animal Handling, but the Player's Handbook offers alternative rules for using different attributes paired with skills:
Normally, your proficiency in a skill applies only to a specific kind of ability check. Proficiency in Athletics, for example, usually applies to Strength checks. In some situations, though, your proficiency might reasonably apply to a different kind of check. In such cases, the DM might ask for a check using an unusual combination of ability and skill, or you might ask your DM if you can apply a proficiency to a different check.
For example, if you were to attempt a tricky maneuver on your mount, it might be reasonable for the DM to call for a Strength (Animal Handling) or a Dexterity (Animal Handling) check, instead of a Wisdom (Animal Handling) check, if such a maneuver is more about being able to hold on to your mount through the maneuver. For more details on using different attributes with skills, see my answer here: Variant: Skills with Different Abilities confuses me