Born in Brooklyn, New York, Gale began playing guitar at the age of 12. Although he majored in chemistry at Niagara University, Gale was determined to pursue a musical career, and began contributing to accompaniments for such stars as Maxine Brown, the Drifters, and Jesse Belvin. He soon began to attract the attention of King Curtis and Jimmy Smith, who began recommending him for studio work. He became known first as a session musician in the 1960s, eventually appearing on an estimated 500 albums. Among the many artists he recorded with were Mose Allison, Aretha Franklin, Bob James, Paul Simon (Gale plays a supporting role in the 1980 film One-Trick Pony, written by and starring Simon), Lena Horne, Quincy Jones, Bob Marley, Nina Simone, Peter Tosh, Grover Washington, Jr., Herbie Mann, Esther Phillips, Joe Cocker, Carly Simon, Van Morrison, Al Jarreau, Dave Grusin, Lee “Scratch” Perry, Paul Douglas and Billy Joel. He also had played in Aretha Franklin’s stage band.