Stoltz grew up in the Detroit area, but eventually found his way to San Francisco after taking a detour to New York City, where he worked in the mail room at Jeff Buckley's management company. Armed with a four-track recorder and a wealth of lo-fi pop songs, Stoltz began recording his own material, performing all the parts himself and drawing comparisons to artists like Brian Wilson and Captain Beefheart. Those songs caught the ear of Monte Vallier, who helped Stoltz clean up and sweeten the recordings for release as The Past Was Faster in 1999. After that, Stoltz upgraded to an eight-track and self-released Antique Glow in a limited quantity of 200 vinyl copies, each one housed in a different, originally designed sleeve by Stoltz himself. Antique Glow was then picked up by Jackpine Social Club for wider release in 2003, which both raised his profile and allowed him to quit his teaching job.
Stoltz found a more permanent home for his music in 2005, when he signed with Sub Pop and released the Sun Comes Through EP. A full-length album, Below the Branches, followed in March 2006. Stolz's band toured that summer as the opening act for the Raconteurs and returned to the studio in 2008 to record his lushest production yet, Circular Sounds. Two years later, he toured with Echo the Bunnymen and released another layered pop album, To Dreamers, which featured his live band on two tracks. His affiliation with Sub Pop having run its course, Stoltz next spent time behind the scenes as a member of Sonny the Sunsets and produced albums for the Mantles and Tim Cohen. He returned in 2013 with an album for Third Man called Double Exposure.
In 2015, Stoltz found a new sponsor in Castle Face Records, the label founded by John Dwyer of Thee Oh Sees. They released In Triangle Time, a set of songs that blended Stoltz's love of the music of the '60s and '80s, in early November 2015. That same year he collaborated with Sarah Bethe Nelson on her Fast Moving Clouds album and also released an set of trippy acid freak-outs under the name Willie Weird called The Scuzzy Inputs Of....
His next musical move came as a bit of a surprise to anyone who didn't know that Stoltz had recorded a song-by-song re-creation of Echo the Bunnymen's Crocodiles album in 2001, or that he once had an Echo cover band with Spiral Stairs called Crockodials. In 2016, he joined the Bunnymen as a touring guitarist, playing the songs he grew up loving. That '80s influence became even more evident on Stoltz's next album for Castle Face, 2017's Que Aura, as he explored synth pop and space disco. He quickly returned with Natural Causes. The record eschewed most of the synths and '80s sounds in favor of a gently psychedelic feel reminiscent of his early work. It was released by the tiny Spanish label Banana Louie in the middle of 2018. After leaving the Bunnymen's employ and suffering the loss of his father, Stoltz's 2019 album My Regime, followed the same template as Natural Causes, but with some added emotional content. Of his two albums released in 2020, Ah! (etc) was very much in that vein while Hard Feelings saw him digging into short, snappy mod songs influenced by the Undertones and Television Personalities. When he returned in 2022 with The Stylist, his second record for the Agitated label, Stoltz was at a stylistic crossroads, harkening back to the piano-based, power pop-influenced sound of his earlier work while also adding woozy soft rock synths and some garage rock as well. ~ Sean Westergaard & Tim Sendra, Rovi