In a horror themed campaign with a lovecraftian feeling, a big part of the game is to be in a constant feeling of being threatened, but without any visual stimuli other than glimpses, or sounds roughly heard of things seething in darkness, and with investigation a big part of the game, contrary to fighting.

Having read Heroes of Horror, I am searching for ideas on how to actually motivate players to assume the role of investigator rather than executor, and how to instil the horror of the unknown, the horror of something that threatens but is not substantial, that cannot be seen or heard, but is there, and has malevolent intentions.

I am thinking of D&D 3.5 as a system, but I am interested of answers that cover any system (Call of Cthulhu is unfortunately out of the question), more as role-playing and storytelling techniques rather than mechanical ones.