Behind the Scenes
Peterson developed a spoken language for the White Walkers to use in Season 1 (probably in the unaired pilot episode), but it was not ultimately used on-screen, and dropped entirely from Season 2. Instead, actual ice-cracking sound effects were used when the White Walkers are conversing.
In an interview with Entertainment Weekly during Season 2, Executive Producers David Benioff and D.B. Weiss stated that "Skroth" is indeed the official name of the language used by the White Walkers. According to the interview, the inhuman (and unpronounceable) sounds of Skroth, "ice-cracking chatter", were created by Sound Designer Peter Brown. 
In the books
The novel A Song of Ice and Fire clearly states that "the Others" ("White Walkers" in the TV series) speak a language with the voice "like the cracking of ice on a winter lake", but the name of the language is not given.