Could un-attended cat familiar cast prestigiate to open a door to go outside?
No, a witch familiar works as a bridge between the dark forces that grants her powers and whispers to her secrets from beyond. In game terms, the familiar works as her spellbook and as her Familiar. As such, the familiar cannot cast spells, as the familiar is not a spellcaster.
Some Advanced Familiars are exception to this, and might have spellcasting abilities, but they use their own abilities and not the spells stored inside the familiar. A sprite familiar has Dancing Lights as spell-like ability for example, while an imp familiar can become invisible at-will. On the other hand, a ratling familiar could conjure spells from a scroll freely, among their own spell-like abilities.
The party encounters an NPC witch's familiar while the witch is away. They first ask it questions using speak with animals, then attack it. What can the cat do?
It can do a lot of things. Firstly, the familiar can do anything that a cat could do, from attacking back to running away or hissing at them. Secondly, her cat is also a familiar, and has all the abilities that do not depend on her master (like deliver touch spells). Depending on her master's level, she could communicate her anger from this transgression to her master mentally (available at 1st level), or even rally all cats in the vicinity against the party on an epic battle against a cat army (available at 7th level), with the proper Diplomacy checks.
What options are available and what the familiar could do is really on the GM's hands. A 1st level witch's familiar has 6 Intelligence (much higher than an animal's INT of 1-2), which goes higher as the witch increases in level. We have a few examples of the capacity of similarly intelligent creatures:
- 6–7: Dull-witted or slow, often misuses and mispronounces words;
- 8–9: Has trouble following trains of thought, forgets most unimportant things;
- 10–11: Knows what they need to know to get by;
- 12–13: Picks up new ideas quickly and learns easily, knows a bit more than is necessary, fairly logical;
- 14–15: Can solve most problems without even trying very hard, able to do math or solve logic puzzles mentally with reasonable accuracy;
With this, the GM should have a general idea of could a certain creature do based on their Intelligence score when not being assisted by their master.
A party of witches get knocked out and imprisoned separately from their familiars. It is up to the familiars to escape, and rescue their masters — can we roleplay the familiars like this?
That actually sounds like a pretty good idea for an adventure if that scenario happens to the players and not their enemies.
Anyway, the familiar has an Empathic Link with her master for up to 1 mile away, they know what kind of emotions each other is feeling. They cannot communicate words mentally, but they can use emotions to pass information to each other. Like seen on these examples on tips about plot hooks with familiars:
Remember that a familiar has an empathic link to its master, and its animal instincts can lead to plot hooks. For example, a toad familiar might project feelings of hunger whenever a member of a fly-demon cult is nearby, a bat familiar might express curiosity about the words a weird hermit is muttering under his breath, and a rat familiar might feel fear when a dangerous assassin walks into the room. A more powerful familiar can speak with other animals of its kind, and if left to roam, it could pick up interesting news about a town or an army camp.
If this location where the witches were taken to is known to them, like the inquisitor's dungeon, they could express their dread towards the place and the familiar could take the hint and figure out where they are, or ask help from those who could know and possibly help them.
The master has the same connection to an item or place that his familiar does.
A cat familiar, if intelligent enough and able to speak with animals of her kind (cats) could Gather Information around town to discover where people in chains are taken to (prisons) or ask where a certain looking witch was taken to based on her physical description. An even more intelligent familiar, like an Imp, could simply fly around town while invisible listening to gossips about where the inquisitors have taken the witches before they are sent to their trials and executed.
What each familiar can do depends on how intelligent it is and how her master has trained them to communicate their emotions, and most importantly, what the GM believes that will make up a better story.