The Finnish language made an impression on Tolkien and was his major inspiration for the Elvish language of Quenya.  The Kalevala is a collection of Finnish mythology, previously a purely oral tradition, put into the form of a song cycle in the early 1800s by Elias Lonnrot.  Tolkien said of the Kalevala, "The more I read of it, the more I felt at home and enjoyed myself."  Upon teaching himself Finnish he remarked that "It was like discovering a wine-cellar filled with bottles of amazing wine of a kind and flavor never tasted before.  It quite intoxicated me."  Parallels will be seen between the Finnish hero Kullervo and Tolkien's Turin Turambar.

ASIN: B00000378I
 

Much of the mythology of Iceland is recorded in the poetic Eddas.  Tolkien was quite familiar with the Icelandic Eddas and in fact borrowed several of the names for the dwarves that appear in The Hobbit from the Voluspá (The Prophecy of the Seeress).  The names of Gandalf (Gandalfr) and Oakenshield (Eikinskjaldi) also came from the Voluspá.

The prose retelling of the Eddas was compiled by Snorri Sturluson in 1220.  Tolkien was familiar with the prose Edda as well as the poetic Edda.
 

Performed by the early music ensemble Sequentia, this is a musical restoration of the Elder Edda texts.  Sequentia is best known for their complete cycle of the music of Hildegard von Bingen.  Edda--An Icelandic Saga is one of their most interesting and beautiful recordings.

Sung in Icelandic by Sveinbjorn Beinteinsson.
 

Beowulf was, of course, of paramount importance to Tolkien in his career as a philologist and professor of Anglo-Saxon and English Language and Literature.  He taught it to his students.  He recited it at lectures.  He published two essays on the subject, and he translated the poem from the Old English himself.  He changed the way other scholars thought about Beowulf.  Until Tolkien's own translation is published (if it ever is), we must be content with what is available.  This is a new translation by Seamus Heaney.