I've read some other posts on here about how to telegraph danger to my PCs and how to make them flee. The problem is, I believe I am actively using all the strategies in both of the posts and my PCs are never afraid of anything. It takes away the suspension of disbelief for me as a DM.
My goal is to make my PCs truly afraid of whatever is coming for them. It's just unrealistic when they are having trouble killing something they see everyday and then a giant CR 11 Retriever steps out of the woods after killing 3-4 treants and starts shooting lasers out of its eyes. It turns one of their NPC escorts into stone and continues to start taking down players. I had PLANNED for them to start running when they saw this. Any sane person would have if they saw that. But my players do not roleplay fear. I absolutely hate this. It's just not what would happen.
Now I'm not saying my party isn't challenged. To be fair, they won that encounter by protecting their escort (a child which they turned invisible) and only lost 1 PC, who got reincarnated by their druid. In other words, they used up ALL of their resources and it ended up being a pretty fun battle. However, they didn't flee.
I also don't mean to say high CR equals more fear. I put a pit in a lower level dungeon with a bunch of sleeping zombies, so when I player fell in I would say, "You feel a hand grasp your ankle and begin to pull you under the sea of dead bodies." That's the kind of thing I'm talking about. But in that situation (which never happened), I'd just expect my player to pull out his sword and begin hacking and slashing. I would much rather for them to have screamed and panicked. Like when a bee flies around your head and you spasm to get it away. If a bee flew around my players' heads - guess what? CR0 bee, smash.
Some further background information: My PCs have already went through a full party wipe and they all know that if they go into certain areas, I'm not lowering the CR for them. PC death is common. They are fine with this and seem to like it because it's more real. Also, my games go well for the most part. We all have fun, I'm just looking for that extra dimension of fear. The only problem is they all seem to have backup characters ready because of this. Maybe that's a bad thing?
So what is an effective way to present my encounters so that my players will respond with a little less "I'm an unstoppable hero", and more "Holy crap! A dragon ahhhh run!!"?