Unless you and the players speak Elvish, you have three options:
- Say that they're speaking another language without saying what it is.
- Say that they're speaking Elvish.
- Say a few Elvish words for flavor.
To decide which option to use, think about the effects of each:
- The party only knows that this language is one they themselves don't speak.
- The characters recognize it and can tell which language it is.
- The players get a bit of flavor, and might be able to recognize it if they've heard it before.
Think about your experience with hearing other languages. Have you ever heard German spoken? (Assuming you don't speak German.) You could describe the experience by simply saying it sounded like German and assume your listeners know what you mean. If the party members are likely to have heard Elvish before, you can assume they know what Elvish sounds like.
DM: The bearded man leans in and says a few words to the Baron. You can't understand what he's saying, but it sounds like it might be Elvish.
If you want to actually say some Elvish words to give them a feel for it, you have two further options:
- Use someone else's Elvish.
- Make up a few words yourself.
A search for "Elvish language" will yield plenty of hits. You could take a few words from someone else's Elvish and say them to the party.
The other option is to make it up yourself. Making an example sound reasonable is more in the scope of linguistics (for which there's an excellent SE site), but I'll give you a few ideas right here. The "feel" of a language is mostly determined by two things: which sounds a language uses and what order they allow them to occur in.
For example, let's say we have a language that uses these consonants: /p t k sh ch y w r/ and these vowels: /u o i/. Next we decide that it allows only syllables that start with a single consonant or one of these consonants: /p t k/ followed by one of these: /y w r/. Let's say words can end in any consonant, and vowels can't clump up. The result allows words like these:
tyiwor, shutosh, wukwop, chokrut
Now we have just enough rules to make up some nonsense words with a coherent feel.
DM: The bearded man leans toward the Baron and says something starting with chukyosh tu kurot, or something like that. He goes on speaking for a minute or so. The word kurot comes up a lot.
If you're going for this option, making up a little bit of language to use, be careful -- some players will assume that there's a language puzzle here to figure out, and they'll start taking notes and try to work out the grammar. Other players won't do anything with it at all -- one bit of random gibberish sounds much the same as any other. Find out what your players like and do more of that. If they have a lot of fun figuring out a language puzzle, go ahead and make up a whole Elvish language if you feel like it. If they don't care about language stuff at all, next time they meet some elves, just say "They're speaking a foreign tongue. Sounds like Elvish." and leave it at that.