Largely self-taught, Montgomery was an important blues pianist with an original style. He was also versatile, working in jazz bands, including larger ensembles that used written arrangements. He did not read music but learned band routines by ear—once through an arrangement and he had it memorized.
Montgomery was born in Kentwood, Louisiana, a sawmill town near the Mississippi border, across Lake Pontchartrain from New Orleans, where he spent much of his childhood. Both his parents were of African-American and Creek Indian ancestry. As a child he looked like his father, Harper Montgomery, and was called Little Brother Harper. The name evolved into Little Brother Montgomery, and the nickname stuck. He started playing piano at the age of 4, and by age 11 he was playing at barrelhouses in Louisiana. His main musical influence was Jelly Roll Morton, who used to visit the Montgomery household.