My character has just taken the Alchemist theme and can now make an item of his level or lower for free using a case of chemicals that costs 50gp (IMO, this theme is the only way to play anything like proper alchemist character). Can anyone tell me in which books or editions of Dragon or Dungeon I might be able to find lists of stuff that he could make? I am particularly interested in stuff that would help a rogue and a party that has no dedicated magic users (unfortunately). Most of the stuff I would like to consider are things that I could craft at level 1, or convince my DM are too low powered to be a higher level.
The DDI Compendium can give you a comprehensive list.
Even if you don't have a D&D Insider account, the Online Compendium will let you do a category search for Items (Alchemical Item). It won't show you the mechanical information unless you have an account, but will tell you the name/level/cost/rarity of each item and its publication source. There are currently nearly 80 such items, most of them below level 10.
Alchemy is underwhelming and expensive
Sadly, I'm afraid you'll find 4e Alchemy doesn't accomplish quite what you're hoping for. It's quite flavorful, but the effects are generally minor and short-lived for the price tag, even if you convince your GM to allow crafting of higher-level items. As utility for a rogue you might find some useful items. (I'm not sure what you mean by filling in for "dedicated magic users," as 4e classes don't really fall into roles that way.)
On the plus side, Alchemy Cases cost 40g, not 50.
A quick troll through the Compendium shows there are a variety of sources for the currently 79 Alchemical Items. The main ones are the Adventurer's Vault, Mordenkainen's Magnificent Emporium and the Eberron Player's Guide. There are also a few items in the Dungeon Survival Guide, and a number in various Dungeon and Dragon issues.