He was assigned to the United States Army Air Force's radio service. After the war his career improved, and he quickly procured his first film role as an inmate in Brute Force. His other film roles include The Naked City (1948), All My Sons (1948), Calamity Jane and Sam Bass (1949), Panic in the City (1968), In Search of America (1971),A Wedding (1978) and No Way Out (1987).
Duff appeared in a number of films with his first wife, actress/director Ida Lupino. One of his later performances was as Dustin Hoffman's attorney in the Academy Award-winning Kramer vs. Kramer (1979).
On radio, Duff played Dashiell Hammett's private eye Sam Spade from 1946 to 1950, starring in The Adventures of Sam Spade on three different networks - ABC, CBS, and NBC. In 1951 Steve Dunne took over the role of Sam Spade.
Duff also appeared on television, having been featured in a 1955 episode of Climax! entitled "Escape From Fear". About this time, he was cast on the religion anthology series, Crossroads.
From January 1957 to September 1958, he appeared with his then wife, Ida Lupino, in their CBS sitcom, Mr. Adams and Eve, which revolves around the private lives of two fictitious film stars named Howard Adams and Eve Drake. He played the young Samuel Langhorne Clemens, in his early life in the West as a satirical and crusading journalist, in the TV series Bonanza ("Enter Mark Twain," season 1, episode 5, 1959). In 1960, he played the male main character in The Twilight Zone episode "A World of Difference" as Arthur Curtis/Jerry Raigan. From October 1960 through April 1961, Duff played Willie Dante, owner of the fictional San Francisco nightclub, Dante's Inferno, in the NBC adventure/drama series Dante.
In 1964, Duff guest-starred as Harold Baker in the episode Prodigy of NBC's medical drama about psychiatry, The Eleventh Hour, starring Jack Ging and Ralph Bellamy. He was then cast as Joe Stillman in the 1965 episode "Mountains to Climb" of the NBC education drama series, Mr. Novak, starring James Franciscus.
From September 1966 through January 1969, Duff portrayed Detective Sergeant Sam Stone in the ABC police drama Felony Squad with costar Dennis Cole.
In 1977, he guest starred in the Rockford Files episode, "There's One in Every Port"; and in 1990, The Golden Girls's "The Mangiacavallo Curse Makes a Lousy Wedding Present". In the 1980s, he appeared on NBC's Flamingo Road; and Knots Landing and Dallas, the latter both on CBS.
Duff had a tempestuous relationship with actress Ava Gardner in the late 1940s. In 1951, he married Ida Lupino. After he was listed in Red Channels as a communist subversive in 1950, he lost his radio work and might have forfeited his entire career had it not been for his marriage. They had a daughter, Bridget Duff (born April 23, 1952). They separated in 1966 and divorced in 1984. He subsequently married Judy Jenkinson.
Duff died at age seventy-six of a heart attack in 1990 in Santa Barbara, California. He was survived by his daughter, his nieces, his second wife, and his granddaughter.