You're right, it don't sound like him, but "Let The Little Girl Dance" was a full six years later and he's described as "versatile" in various bios. Just had a quick look on google and some of the sources have him as "a member of The Bees..." so perhaps he wasn't actually the lead singer.
Now this is interesting. Michel Ruppli tells me this was recorded in Los Angeles on September 10 1954. However both sides were written by Dave Bartholomew who rarely left New Orleans. And, after a quick internet surf I find the lead singer on these tracks was Billy Bland, best known for his later New York recording "Let The Little Girl Dance".