Efrem Zimbalist was born in the southwestern Russian city of Rostov-on-Don, the son of Jewish parents Maria (née Litvinoff) and Aron Zimbalist (Цимбалист, Russian pronunciation [tsɪmbaˈlʲist]), who was a conductor. By the age of nine, Efrem Zimbalist was first violin in his father’s orchestra. At age 12 he entered the Saint Petersburg Conservatory and studied under Leopold Auer. He graduated from the Conservatory in 1907 after winning a gold medal and the Rubinstein Prize, and by age 21 was considered one of the world's greatest violinists.
After graduation he debuted in Berlin (playing the Brahms Concerto) and London in 1907 and in the United States in 1911, with the Boston Symphony Orchestra. In 1912, he played the Glazunov Concerto in a concert marking Leopold Stokowski's first appearance with the London Symphony Orchestra. He then settled in the United States. He did much to popularize the performance of early music. In 1917, he was elected as an honorary member of Phi Mu Alpha Sinfonia, the national fraternity for men in music, by the fraternity's Alpha Chapter at the New England Conservatory of Musicin Boston. In 1928, Zimbalist began teaching at the Curtis Institute of Music in Philadelphia. He was director of the school from 1941 to 1968. His pupils included such distinguished musicians as Aaron Rosand, Oscar Shumsky, Felix Slatkin, Shmuel Ashkenasi, Harold Wippler and Hidetaro Suzuki.
He retired as a violinist in 1949, but returned in 1952 to give the first performance of the Violin Concerto by Gian Carlo Menotti, which is dedicated to him. He retired again in 1955. He served as a juror of the International Tchaikovsky Competition in 1962 and 1966.
His own compositions include a violin concerto, the American Rhapsody, a tone poem called Daphnis and Chloe, a Fantasy on themes from The Golden Cockerel by Nikolai Rimsky-Korsakov and a piece called Sarasateana, for violin and piano. He also wrote an opera, Landara, which premiered in Philadelphia in 1956.
He married the famous American soprano Alma Gluck and they toured together for a time. Alma Gluck died in 1938. In 1943, having been a widower for five years, he married the school's founder, Mary Louise Curtis Bok, daughter of publisher, Cyrus Curtis, and 14 years his senior.
He died in 1985, at the age of 94. His and Alma's son, Efrem Zimbalist, Jr., and their granddaughter, Stephanie Zimbalist, both became popular actors.