The Tolkien language inspired partly by Finnish is Quenya. Tolkien developed Quenya as a highly structured Elvish language, drawing on Finnish phonology and grammar as well as influences from Latin, Greek, and other languages.
Within Tolkien's fictional history, Quenya is associated especially with the High Elves, including the Ñoldor and Vanyar. It was conceived as an ancient, formal language with great cultural prestige, rather than simply a general language spoken by every Elf in every period.
Sindarin is the other major Elvish language readers often confuse with Quenya. Sindarin became the everyday Elvish language of much of Beleriand and later Middle-earth, while Quenya retained a more elevated and historical role in many settings.
Tolkien's languages were not decorative word lists. He created systems of sound change, grammar, vocabulary, and historical relationship, using his professional knowledge of philology. Quenya therefore contributes to the sense that Middle-earth has a deep, internally consistent past.