You can make this Goat part of your world's Folklore
What you can do is create a new monster, per DMG p. 273-283, with a twist.
What you are doing in this case is, since you are interested in this goat moving up as the PC moves up, is creating an "evolving" monster. You can grow the goat in the same way dragons increase in size and power as they age. (He'll do it faster, of course). That is one template. Since the goat is adventuring, though, it would be worth folding in elements of Creating a new race/species (p. 285 DMG) of Goat or Giant Goat.
Heck, let him grow! Paul Bunyon's ox Babe was larger than average! The trick is to find the sweet spot in the ratio of CR to the player character's level. (see below). As this is homebrew, it may be an iterative process.
Use the point value guide in the MM for various abilities and skills that you give to the goat to bump up its CR as the character goes up in level.
What CR should I shoot for? Good question, as the match up from CR to PC Lvl is not one-for-one.
Consider the Archmage NPC who can cast level 9 spells, and who has a CR of 12, and then consider that a CR of 12 is a suitable challenge of 3 or 4 characters at level 12 ... it all depends on the skills, hit dice, and abilities you give to your goat.
As it levels up, add some features to your goat to bump its CR up:
Bump up ASI. Consider giving it boosts to INT to where it can
nearly talk/communicate via baa's and grunts with the PC.
Damage per attack
- Increasingly effective "knock prone" or "Knock back" melee
- Number of attacks per round.
- Increased perception, be it visual, olfactory, audio, taste.
- Increase resistances to X.
- Increase dexterity (ASI)
- Increase Constitution (ASI)
- Increase Wisdom
When you upgrade the goat's kit/package every few levels, look at all of the features you now have and try to match it to a CR as if it were a monster. If the CR is about 1/4 to 1/3 of the PC's level, you should be in the ball park of not getting it to be too powerful, but still has a chance to survive adventures as the character goes up in level. When your PC is about 18th level, a CR between 5 and 6 ought to fit well enough. That puts him between a Triceritops (10d12, CR 5) and a Mammoth (11d12, CR6) in challenge rating, though he need not be that huge if you don't want him to be. Some of his HD increase can reflect how he's learned to handle danger/combat better.
At some point, this goat will become famous, as it is the companion of a hero, and will be like Babe the Blue Ox. You can decide/rule that the goat has had enough adventuring, and either wants to be put out to stud, put out to pasture, or wants to go to sea. (Every ship has a goat locker where the salty old NCO's and Bosun's mates hang out. A perfect place for a wise and experienced old goat).
Consider the stud fees you can charge for this amazing goat ...
Experience with something like this: it's been a few years, but I had to create an evolving PC/Monster when the dwarf in the campaign I was running, in 1e, died and got reincarnated as a Brass dragon. When that roll came up, I was puzzled at how I'd keep the player interested in continuing on. I was fortunate: the age/hd template was already there, but it was tricky (and we had a few false starts) in keeping Miles from being too powerful compared to the party. As the group never got past 8th level, we never got to see him blossom, but it was instructive nonetheless. 5e has a lot more guidance on how to pull this off.