Just a Call of Cthulhu rulebook and the Delta Green basic book.
My favorite version of the Delta Green book is the bi-system BRP/D20 one (now out of print). I think the D20 rules for Call of Cthulhu (also out of print, but easily available via Amazon) offer a nice format for playing a more action/adventure themed Delta Green game than might be suggested by using the BRP rules, and there are a couple of nice ideas in there that a solidly BRP GM might find useful too.
Usefulness: The DG sourcebook is probably the most impressive setting book produced for Call of Cthulhu. The wealth of stuff in there is astounding. I don't necessarily think everything the authors did with it follows my own vision of The Mythos And How It Works, but it ain't shabby by any means and I highly recommend it whichever rules you want to use.
Other books: There are other materials produced in the DG line, but they aren't necessary to get started, or to play for months. They refine and expand the materials in Delta Green.
Overview? Think X-Files meets Call of Cthulhu.