Rudd's first instrument was the French horn, which he studied from the age of 11. His father was an amateur drummer who introduced his son to jazz. In his teens, Rudd began teaching himself to play the trombone. Woody Herman's star trombonist, Bill Harris, was a particular favorite. He played Dixieland while he attended Yale, with a band called Eli's Chosen Six. From 1960-1962 he worked with legendary pianist Herbie Nichols, who became something of a mentor to Rudd. From 1961-1963, Rudd played in a band with soprano saxophonist Steve Lacy and drummer Dennis Charles. The band would later be informally known as the School Days Quartet, after the 1963 Emanem album of that name. The group's repertoire consisted entirely of Thelonious Monk tunes.
In 1962, he joined trumpeter Bill Dixon's free jazz group, which also included tenor saxophonist Archie Shepp and drummer Charles. In 1964, he founded and co-led the New York Art Quartet (with saxophonist John Tchicai) and participated in the October Revolution in Jazz, an early free jazz festival organized by Bill Dixon and held in a New York City café. Rudd spent the latter half of the '60s playing in Archie Shepp's band, Charlie Haden's Liberation Music Orchestra, and a group led by saxophonist Gato Barbieri. In 1968, he formed the Primordial Quartet (with saxophonists Lee Konitz and Robin Kenyatta and pianist/vibist Karl Berger). The group disbanded in 1970. Rudd's compositions for the Jazz Composer's Orchestra were recorded in 1973 on the album Numatik Swing Band (JCOA).
Although Rudd recorded occasionally in the '70s and '80s (notably under Lacy's leadership), he gradually became less visible as matters of economic survival took precedence over creative concerns. He worked a variety of non-musical jobs and spent time teaching at the college level. After being denied tenure at the University of Maine in Augusta, Rudd moved to the Catskills region of New York State, where he worked steadily in a hotel resort band. By the mid-'90s, he started to record more frequently. Albums on the CIMP label, under his own name and as a sideman, helped reestablish him as a jazz player. At the turn of the millennium, Rudd performed with some frequency in Europe and New York, regaining his reputation as the father of free jazz trombone.
In 2000, Rudd and Lacy reunited with Lacy's regular rhythm section, bassist Jean-Jacques Avenel and drummer John Betsch, and vocalist Irene Aebi, to record Monk's Dream for the Verve label; the band also toured in support of the album. Also during the 2000s, Rudd began delving into various world music projects, including 2001's Malicool, featuring musicians from West Africa, and 2003's Blue Mongol, recorded with Mongolian throat singers. In 2007, Rudd kept the world vibe going with the Afro-Cuban and South American-flavored El Espiritu Jibaro. In 2008, Rudd featured vocalist Sunny Kim on Keep Your Heart Right. A year later, he released the trombone album Trombone Tribe, featuring a bevy of players including Josh Roseman, Wycliffe Gordon, and others. In 2011, Rudd celebrated his 75th birthday with the release of The Incredible Honk. Two years later, he cut a set of pop, country, folk, and blues standards for Sunnyside under the title Trombone for Lovers. It featured a choir as well as individual vocalists, including Heather Masse.
In his 80th year, Rudd collaborated with Masse, a member of the Wailin' Jennys. Enlisting bassist Mark Helias and guitarist Ralf Sturn, they cut a series of originals and jazz standards in intimate, almost informal, recording sessions. The end result, August Love Song, was released by Red House Records in February of 2016. The improvised Strength Power arrived one month later, and in 2017 Rudd returned to jazz standards for the intimate Embrace. In December of that year, Roswell Rudd died at his home in the Catskills at the age of 82. ~ Chris Kelsey, Rovi