The familiar is a tumor while inside your body, not a creature . So, unlike equipment, which are carryed or wielded, the creature is part of your character as much as your arms and legs. Which means that if the alchemist is reduced in size, the tumor's size is properly adjusted.
It can detach itself from your body and become a living creature under your control. So, from the rules as written, you can have a Tiny tumor familiar while being a Tiny creature yourself.
As GM, i would give the player two options:
First, that since you are both two different creatures, you would have to also permanently reduce your tumor familiar, or pick a diminutive familiar instead. Otherwise, whenever you are both merged together, the result would maybe cause some sort of physical penalties, and definitely cause social penalties.
Second, the familiar and the alchemist are one creature while the tumor is attached, so anything affecting the alchemist would affect the familiar. But once the familiar detached from him, it is now a separate creature and none of the spells affecting the alchemist would be affecting the familiar.
(PS: Do not look for images of large tumors on google.)