RAW, you Blink nakedly to the Ethereal Plane
With a strict reading of the rules, only "you" "vanish from your current plane of existence" - so you don't take anything with you.
RAI, you take the stuff you're carrying
This is discussed in If I am holding an item before I cast Blink, will it move with me through the Ethereal Plane?. It seems clear to sensibly play the game that Blink and spells like it will also transport your clothes, held equipment etc.
No creatures come with you
If Blink did something as significant as letting you take a whole other creature with you to the Ethereal Plane, it would say so. Unlike letting you take your clothes and held equipment, taking creatures you're touching with you is not a necessary modification to make the spell sensible (and indeed in situations with enemy creatures could be highly detrimental to the spell's utility).
Your familiar should be ok
At the end of a fall, a creature takes 1d6 bludgeoning damage for every 10 feet it fell
Assuming your shoulder is less than 10 feet above the ground, your familiar shouldn't take any damage from falling off you as you Blink.