I'm a huge sucker for blending cthulu-inspired material into high fantasy. I can't stop myself. What books (and magazine articles) provide background (3rd party material okay). Please specify as much detail as possible for indicating if the resource has enough detail to make worthwhile.
WotC hasn't produced a Far Realms book yet, much to my sadness, and none of their books provide a ton of star pact material. The planar books are more useful for infernal or dark pact warlocks, unsurprisingly.
There are a few good Dragon articles. "Performing the Pact" is a general guide to roleplaying pacts and patrons; it contains hints and one sample patron for each pact, including the star pact. There are also feats and background options appropriate for a star pact warlock. "Wish Upon A Star" is nearly essential for a star pact warlock. It ties the stars into your favorite Cthulhoid entities and provides a ton of roleplaying advice useful for both GMs and players. It's also got a lot of mechanics: feats, lots of powers, a paragon path, and an epic destiny.
The Eberron Campaign Guide is worth looking at for the material on the Cult of the Dragon Below. They're the aberrant-worshippers of the setting. I was particularly amused by the Finches, a family of backwoods darkness worshippers, with names like Thelonius and Lucretia. Lovecraftian if anything ever was. The monsters of Khyber and Xoriat fit Lovecraftian themes nicely as well.
As far as third party material goes, Goodman Games has a book of Cthulhu monsters, Critter Cache 6: Lovecraftian Bestiary. Issue #2 of their magazine, Level Up, has monster stats (which might be duplicated from the Critter Cache book) and a Cthulhu-oriented adventure.
Besides the 4e material, I'd at least think about some of the Lamentations of the Flame Princess books. The main rulebook/boxed set probably isn't that applicable, but I have Death Frost Doom and some of his Pembrooktonshire material: it's all very creepy and not tightly tied to any one system. Tower of the Stargazer has a very promising title.