It gets the benefits immediately
You get the Companion's Bond feature at 3rd level, at the same time as you gain the Animal Companion feature.
The Companion's Bond feature says:
In addition to the areas where it
normally uses its proficiency bonus, an animal
companion also adds its proficiency bonus to its
AC and to its damage rolls.
This applies from 3rd level and onwards, so it gets a +2 to its attack rolls, ability and skill checks, saves, AC and damage, and loses Multiattack if it had it.
Once you reach 4th level, and every level beyond that, it gets an extra hit die:
For each level you gain after 3rd, your animal
companion gains an additional hit die and
increases its hit points accordingly.
This means that your giant weasel's stats look like this:
Medium beast, unaligned
Armor Class 15 (13 +2 proficiency)
Hit Points 9 (2d8)
Speed 40 ft.
(+2 proficiency to all saves)
STR 11 (+0)
DEX 16 (+3)
CON 10 (+0)
INT 4 (-3)
WIS 12 (+1)
CHA 5 (-3)
Skills Perception +5, Stealth +7
Senses darkvision 60 ft., passive Perception 13
Challenge 1/8 (25 XP)
Keen Hearing and Smell: The weasel has advantage on Wisdom (Perception) checks that rely on hearing or smell.
Bite: Melee Weapon Attack: +5 to hit (+3 +2 proficiency), reach 5 ft., one target. Hit: 7 (1d4 + 3 + 2 proficiency) piercing damage.