Born in Goodman, Mississippi, Davis was raised in Chicago. He graduated from DuSable High School before studying at the Chicago School of Music. Davis also studied privately with John Hauser. During the 1950s, he played with Billie Holiday, Ben Webster, Sun Ra and Dinah Washington. Davis also performed and recorded with Kenny Dorham, with whom he associated musically for many years.
During the 1960s, he performed and recorded with Elvin Jones, Jimmy Garrison, Illinois Jacquet, Freddie Hubbard, Johnny Griffin, Steve Lacy and Ahmad Jamal, also working with Blue Mitchell, Erskine Hawkins, John Coltrane and Clifford Jordan. In 1964, Davis topped Downbeat Magazine‘s International Jazz Critics Poll for baritone saxophone. He performed in the musical The Philosophy of The Spiritual – A Masque of the Black with Willie Jones, produced by Nadi Qamar. Davis taught at PS 179 in Brooklyn and was musical director of the Turntable, a nightclub owned by Lloyd Price.