Recipient of four Emmy awards, Rose was born in London, to Jewish parents, and raised in Chicago, Illinois. Rose's career in music began when he worked with Ted Fio Rito's band when he was sixteen. Rose also worked as a standby pianist for NBC Radio. It was here, in the early 1930s, that he first gained a reputation, while arranging for the Frank Trumbauer orchestra and later leading a house band at station WGN. He composed several early swing originals such as "Break It Down" with Frankie Trumbauer, "Transcontinental," "Plantation Moods," and a piece recorded under three different titles: "I've Got It", "Itchola", and "Jigsaw Rhythm," his original version with the WGN band including Louis Prima.
Rose was asked to come to Hollywood, where he formed his orchestra, doing a twice-weekly radio show for Mutual Broadcasting System calledCalifornia Melodies, where he wrote all the broadcast arrangements. He worked his way up to becoming music director of the Mutual network. Rose's first try at composing was his hit song "Holiday for Strings". During World War II, Rose entered the Army and it was here where he met Red Skelton. Skelton asked Rose to become the conductor for his Raleigh Cigarettes Program. Rose joined the cast in 1948 and went on to work with Skelton on his television show for over 20 years.
In 1957 his rendition of Larry Clinton's "Calypso Melody" became Rose's second million selling record, and was awarded a gold disc.
"The Stripper" was composed by Rose and recorded in 1958. It was originally used as the B-side to his single, "Ebb Tide". The choice of the record's B-side was not by Rose, but by an MGM office boy. MGM indicated they wanted to put the record on the market quickly. A B-side was needed and with Rose away, the office boy went through some of Rose's tapes searching for one. "The Stripper" featured especially prominent trombone lines, giving the tune its lascivious signature, and evokes the feel of music used to accompany burlesque striptease artists. The piece features in the films Slap Shot, The Full Monty and Wallace & Gromit: The Curse of the Were-Rabbit as well as TV series Little Britain and Scrubs. It was also famously used in a parody by British comedy duo Morecambe and Wise, where they danced to the tune while making breakfast.
"Holiday for Strings" became well known as the theme for Red Skelton's programs. A parody version, retitled "Holiday for States", was recorded as a vocal by Allan Sherman, with the straight melody but with ersatz lyrics comprised solely of the names of the American states. In Rowan and Martin's Laugh-In, Ruth Buzzi "laughed" the melody of the song, imitating the Spike Jones version. On his album Leon Live, Leon Russell plays a strain from the song in the middle of a performance of his own song, "Shootout on the Plantation." The fast-paced song was featured in a shopping scene in the Bridget Fonda-Nicolas Cage film "It Could Happen To You".
He had two daughters with his third wife, Betty Bartholomew. His granddaughter is singer-songwriter Samantha James.He was married on October 8, 1938, to the actress Martha Raye. They were divorced on May 19, 1941. Rose was married for a second time, on July 28, 1941, to the actress and singer Judy Garland. They had no children, though Garland, according to biographer Gerald Clarke, underwent at least one abortion during the marriage, at the insistence of her mother, her husband, and the studio that employed her, MGM. Garland and Rose divorced in 1944.
Rose was a live steam hobbyist, with his own backyard railroad.
Rose died in Burbank, California at the age of 80 and was buried in Mount Sinai Memorial Park Cemetery in Hollywood Hills, California.