I realize the question is just about RAW on the Pact of Chain Familiar, but there were some other things in the question that I felt should/could be addressed.
Rests
Limiting the number of short rests is a tactic some DMs use to stress resources, typically they find reasons to interrupt the rest. The DM can have any house rule to limit resting also, but doing so really messes with the balance of the Warlock class.
Pact of Chain
Familiars are capable of telepathic communication with the caster, by way of the spell:
While your familiar is within 100 feet of you, you can communicate with it telepathically.
As other answers say, the Imp, RAW, do speak both Common and Infernal.
The spell reads:
[T]he familiar has the statistics of the chosen form... (Find Familiar 240 PHB)
And the Imp stat block says:
Languages Common, Infernal
Person Problem
Your post makes it sound like you feel there is also some unfair nerfing of your character that is really the root of the question. I would suggest that you respectfully approach your DM and communicate that you feel like you're being unfairly penalized. Approach it from the standpoint of trying to reach an understanding.
From a balance perspective: The Warlock has the second most powerful free attack in the game (Eldritch Blast, not counting Hex), but they only have a few spell slots. They recharge on a short rest, but those are limited by activities that have to get done. Those slots max out at 5, not counting the Mystic Arcanum.