There are three ways by which to measure appropriate in this context; RAW, emulative, and challenging.
The animate dead spell says (emphasis mine):
Choose a pile of bones or a corpse of a Medium or Small humanoid within range. Your spell imbues the target with a foul mimicry of life, raising it as an undead creature. The target becomes a skeleton if you chose bones or a zombie if you chose a corpse (the GM has the creature's game statistics).
Skeleton does in this context mean the specific creature, which is a Medium sized undead with a CR of 1/4. It may be worth noting that assuming babies are Tiny (DM's call) their bones is an invalid target of animate dead (for a lot of tables I would think this is completely fine).
If we allow some fudging of the rules (as is a DM privilege) and we simply want to find a representation for what this should be in the world (i.e. irrespective of the challenge to a party; see below) we can most easily accomplish this through modifying the skeleton stat block. There is no right answer for which/how many of these modifications to make, so it's going to up to you.
(The full guidelines for creating/modifying can be found in the Dungeon Master's Guide p. 273.)
The most obvious thing to change is the creatures size: from Medium to Tiny. The only rule relating to this is its hit die changing from d8s to d4, but if give it one more hit die (to 3) it keeps the 13 hit points and its CR is not changed.1
The second change would most likely be to replace the weapons it has with a claw attack. If you let the baby skeleton use Dexterity for the attack and deal 1d6 slashing damage (+ Dex) you are not changing its expected damage and therefore not its CR.
Third, something that crawls on the ground might be considered slower than something that can walk, so you might want to lower its speed (say from 30 to 20). Minor changes to speed should not affect a monsters CR according to the DMG guidelines.2
If you think a baby skeleton should be less dangerous than an adult skeleton (considering both are raised using the same amount magical energy; i.e. same spell) you need to lower its CR. We might also want to make it more dangerous than a crawling claw (CR 0) because its just a hand (and we have a whole baby). We only have one option for CR and that is 1/8.
There aren't any sure-fire way to do this accurately, but lowering either the damage or hit points of the skeleton should be enough, so you might just want let it keep 2 hit dice from the above consideration and call it a day.
A singular CR 1/4 creature isn't a threat to a even a 1st-level party. So if you want this encounter to be more challenging you need to find a CR which would be appropriate for your party3 and make modifications such as the ones above to fit it as the creature you want. However making a higher CR skeleton puts us thoroughly into the realm of homebrewing which means there are too many factors and options for this answer to address them appropriately. If you do go down this route and you are uncertain you've gotten it right you can come back and ask if you've gotten it right (as a new question obviously), making sure to follow our homebrew review guidelines.
1: There are no rules associated with the number of hit points a monster has, or more accurately they have a number of hit dice so that they have the appropriate/desired number of hit points.
2: Obviously if has no speed, or a speed such that one or more party members can't get to it (in particular flying) its CR might have to be adjusted, but how much is difficult to determine accurately (read: to be done in playtest) as it will vary heavily with party composition. A party with range will be under little to no threat from something that can't move, and a all melee party will have no chance against a creature with flight and ranged attacks. The environment is also a factor to move speeds and inside non-massive rooms a change from 30 to 20 walk speed is completely ignorable.
3: For four adventurers it is nominally equal to their level, though you may wish to look at the table on page 88 of Xanathar's Guide to Everything. Also note that as always you might need to adjust CR depending on party optimization/playstyle, specifically appropriate features (such as Turn Undead), and magic items and similar.