Elvish is the language, elven is the adjective
Are there rules on when to use elf (as an adjective) versus elvish versus elven in D&D?
In D&D 5e, at least, there actually are!
The official D&D Style Guide can be found as part of this style guide resource pack on DMsGuild. The latest version is v1.08a, though an older version (v1.04a) of the PDF can be found online here.
On p. 9 of the latest version (or p. 7 of the v1.04a PDF), it provides a word list, with parentheticals occasionally describing what the word refers to or how it is to be used. Of note, these two entries appear:
An earlier section also notes that language names are always to be capitalized.
In particular, this word list makes it clear: The term "Elvish" should be used to refer to the language, and "elven" should be used as an adjective to refer to anything of or relating to elves. The word "elf" is a noun, not an adjective (though it occasionally appears as part of a compound noun).
(The same distinction exists for dwarves; "Dwarvish" is the language, "dwarven" is the adjective.)
Examples of the usage of both terms can be found in 5e's descriptions of the elf race. For instance:
Your elf character has a variety of natural abilities, the result of thousands of years of elven refinement.
Although elves reach physical maturity at about the same age as humans, the elven understanding of adulthood goes beyond physical growth to encompass worldly experience.
In addition, every elf bears a family name, typically a combination of other Elvish words. Some elves traveling among humans translate their family names into Common, but others retain the Elvish version.
The Languages trait even uses both:
You can speak, read, and write Common and Elvish. Elvish is fluid, with subtle intonations and intricate grammar. Elven literature is rich and varied, and their songs and poems are famous among other races. Many bards learn their language so they can add Elvish ballads to their repertoires.
Here, it describes the Elvish language and ballads in the language, but speaks of the literature of elves in general.
I don't know whether previous editions perfectly follow this same convention, but I imagine that they largely do adhere to similar conventions in this regard.