Nerfing the Sleep spell, without warning, is a jerk move
In another answer, CaptainBumbleFudge mentioned that the Unconscious status should cause the ankheg to "drop whatever it's holding". But the way the rules handle this situation is actually even more specific than that. To run through it all in order:
The Sleep spell states:
...each creature affected by this spell falls unconscious...
The description of the Unconscious condition states:
An unconscious creature is incapacitated...
The description of the Grappled condition states:
The condition ends if the grappler is incapacitated.
These rules interactions are clear. You and your fellow players are right to expect the Sleep spell to cause an affected creature to release a grapple. (Especially when we're talking about a very ordinary, low-CR monster, which you wouldn't expect to be an exception to the rules.)
You're right to be dissatisfied with the result your DM gave you...
...because it doesn't seem to respect your spellcaster's choices as a player. The spellcaster in your party only has so many opportunities to add a new spell to their repertoire, only so many spell slots between rests. They chose to take a spell that has a specific kind of utility, and they chose to use a spell slot when they saw an opportunity to solve a problem. And that was a smart choice, because it's a problem that the rules explicitly state that this spell will solve!
I'm sure that your DM enjoyed surprising the party with a monster that was lying in wait during your previous fight. Setting aside the question of whether this monster could reasonably surprise the party, it should be dramatic in a rather cinematic way. If presented well, it would hopefully get the party fired up to beat this monster. But for some reason, your DM wasn't satisfied when the party came up with an effective solution to this challenge.
If your DM likes presenting the players with unexpected challenges, and is eager to houserule to accomplish that, then he should be prepared for unexpected solutions. Hopefully that's why he's throwing these challenges at you in the first place.
If he likes to be adversarial, and is eager to houserule to get an advantage over the players, then you're in real trouble. After all, a DM can houserule whatever they like, and ensure themself the advantage in every situation... but that's not much of a game. If it really seems that way, then you just gotta tell him, "It seems like you change things just to give yourself an advantage". Maybe he needs to know that's not fun for everyone, or maybe he doesn't realize it comes off that way.
But I think it's more likely that your DM simply didn't like the party's solution. He may be thinking "the Sleep spell shouldn't make this so easy", or "putting monsters to sleep isn't interesting". That's a bad attitude for a DM to have. A player invests in their character's ability to accomplish specific tasks, and generally, a DM should reward this. This isn't a case of an overpowered ability trivializing an encounter. It's a basic spell versus an ordinary monster, in a situation where the rules and the dice dictated that the PCs should get the result they were aiming for.
If that's the case, he might need to keep that in mind, so that players' effort doesn't seem wasted. He should be attentive to situations where the players are using up their resources, or making choices at the cost of missing other opportunities.
But he might also be motivated to provide challenges that the rules DON'T prepare you for! In that case, he should make it a more robust element of the game, not merely a tool for him to use against you.
If your DM likes to improvise stuff like this, he should make it fun
The changes you describe your DM making to the ankheg seem reasonable. It's silent while burrowed underground, and its jaws lock when grappling. They're relatively small modifications, and they fit a sort of fantasy verisimilitude. This could be a lot of fun - if your DM makes it something you can interact with, instead of pulling it out without warning.
It seems like there were a lot of opportunities for your DM to provide a "teaser" for these aspects. If you had a chance to observe the first ankheg before fighting it, he could mention that it's eerily silent in its movement. Or if it was an abrupt confrontation, he could mention that it very nearly got the drop on you, but wasn't quite burrowed in for its ambush.
Once the ankheg grappled your character, he could mention right away that its jaws seem to be locked. If he tells you there's clearly no way you could possibly wriggle free, not without overpowering it or just hacking your way out, this would still be a surprise. But it would be presented as a challenge to face outright, not a "gotcha" that negates your effort after you've already tried to react.
This can be a matter for flavorful description. He could mention that these particular ankhegs have unusually massive and powerful mandibles, or rigid, overgrown carapaces that click like manacles whenever their jaws twitch. If you figure out that they're gonna get you in a vise grip, he can still say "nuh-uh, Acrobatics won't get you out of this one". But once you anticipate that he's made one of its attacks stronger, that lends some additional tension to the fight.
It could be a matter for PCs' skills - Maybe one character logically should've heard all about every ankheg that ever rampaged across someone's farm back home, and would notice something weird about these. Good place to call for a Nature or Survival roll! A poor roll might mean he won't tell you anything more than that, but it would be a good way to let you know "Hey, I'm gonna play some kind of trick on you with this monster", and let you try to anticipate it.
If your DM thinks these changes to the monster are clever (which is hopefully the case; I presume he doesn't think they're cheap tricks), then he can probably come up with a clever way to communicate them. If the players figure it out, everyone feels smart, and the monsters are still tougher than usual. If the players miss the clue, then he can point it out later, and you learn a bit more about how he imagines these aspects of monsters.
Any of these techniques require some forethought, rather than improvising right in the middle of the fight... but that's necessary to make this something the players interact with, instead of something that interferes with players' success.