Best known as the in-house producer and proprietor of the Shimmy Disc label, Kramer (first name Mark) has also had an extensive career as a musician, often in bands or with collaborators, but occasionally solo as well. His music tends to be atmospheric with a touch of the abstract, often using found sounds to add texture and detail to his recordings, and his themes are often deeply personal, dealing with issues in his life and his own creative fascinations, though not without a sense of humor. 1992's The Guilt Trip was his first magnum opus, an eclectic two-hour spectacular; 2012's The Brill Building paid homage to great Jewish pop songwriters; 2021's And the Wind Blew It All Away was a meditation on love in difficult times; and 2022's Music for Films Edited by Moths was a set of atmospheric, minimalist compositions.

Born in New York City in 1958, Kramer initially chose film school as his creative path, but wound up dropping out twice. Instead, he worked as a sound engineer for several late-'70s New York bands and eventually joined the band that became Eugene Chadbourne's Shockabilly as a bass player and sometime-organist. Upon Shockabilly's mid-'80s dissolution, he served a brief stint as the Butthole Surfers' touring bassist, and in 1985 opened a 16-track recording studio, Noise New York, that became a hub for the city's underground music community. In 1987, he founded Shimmy Disc Records, an independent label that was home to dozens of maverick acts, including Dogbowl, Ween, Daniel Johnston, Fred Lane, and the Boredoms. Kramer produced acts ranging from the label's own GWAR and King Missile to Galaxie 500, Urge Overkill, Half Japanese, Daniel Johnston, Fred Frith, Palace Brothers, Low, and Royal Trux.

From the late '80s onward, Kramer was a highly active collaborator, releasing projects with Jad Fair of Half Japanese, John S. Hall of King Missile, Ralph Carney and Daved Hild, ex-Gong frontman Daevid Allen, ex-King Missile member Dogbowl, ex-Soft Machine bassist Hugh Hopper, and Captain Howdy, a band featuring magician Penn Jillette on vocals and cartoon voice-over master Billy West (Ren & Stimpy, Futurama, etc.) on guitar. Kramer also closed Noise New York and opened a 24-track facility across the river he dubbed Noise New Jersey. After the breakup of Bongwater (which ended in a contentious lawsuit), Kramer's solo career began in 1992 with the release of The Guilt Trip, followed in 1994 with The Secret of Comedy and 1995's Japan-only Music for Crying; the albums combined Kramer's warped humor with his ample production skills and (sometimes) pop songwriting sense. In 1998, Kramer returned to solo recording with Songs from the Pink Death on Shimmy Disc and Let Me Explain Something to You About Art on Tzadik. The Sound of Music, his second album with Jad Fair, appeared the following year.

In 1999, after an unsuccessful European tour, Kramer quit the music business, but his retirement didn't last long. A second Kramer release on Tzadik, The Greenberg Variations, arrived in 2003. In 2004, he opened a mastering studio in Florida and dabbled in writing music for theater while he put his own recording career on hold. He made his comeback as a performer with 2012's The Brill Building, a collection of covers of songs written by Brill Building greats including Carole King, Burt Bacharach, and Neil Diamond, which were given Kramer's distinctive touch; Tzadik issued the LP. Kramer continued to produce albums for artists ranging from Colleen Green to Permahorn, and in 2017 launched the vinyl-only label Shimmy 500 with the Jad Fair collaboration The History of Crying. That same year, The Brill Building, Book Two, with guitarist Bill Frisell, arrived on Tzadik.

Shimmy 500 proved to be short-lived, but he gave Shimmy Disc a proper re-launch in 2020, as a partnership with Joyful Noise Recordings, who named Kramer their Artist-in-Residence for the year. Let It Come Down, Kramer's project with British singer Xan Tyler, released their debut album, Songs We Sang in Our Dreams, during the year. In 2021, Kramer released the limited vinyl box set Make Art, Make Love, Die, which included the Let It Come Down album, two experimental LPs, and two albums which were also issued individually: Words Music, Book One (containing musical adaptations of spoken recordings by poets like Allen Ginsberg and Gregory Corso), and Kramer's first solo collection of original songs since 1998, And the Wind Blew It All Away, in 2021. Music for Films Edited by Moths, an album of ambient pieces originally included in the box set, was given a stand-alone release in 2022. ~ Steve Huey, Rovi