Tim Foljahn, the guiding force behind Two Dollar Guitar, began the band when he landed in Hoboken, NJ, after several years traveling around the country, with brief stops in New Orleans, Alberquerque, and Chicago. Hooking up with his friend, Sonic Youth drummer Steve Shelley, he began to record a collection of dark ballads that he had accumulated during his travels. The result was his debut single, Lost Bird (Smells Like Records, 1995), which was produced by Shelley and featured him on percussion, but regardless was largely a solo effort by Foljahn. Soon after, they released Let Me Bring You Down, also on Smells Like Records, which delved deeper into Foljahn's collection of grim songs.

Foljahn and Shelley's partnership was developed further as they began playing together with other bands. Around the release of Let Me Bring You Down, they put in time with groups like Mosquito, Male Slut (with Thurston Moore), and Cat Power. Their second full-length issued on Smells Like Records in 1996, Burned and Buried pulled in many more guest musicians and reached deeper into folk, jazz, and blues roots, incorporating pedal steel and blues harp to create an album that featured mature instrumental work as much as lyrics. Two Dollar Guitar, in essence, became a whole band with this album as they added ex-Cell bassist Dave Motamed as a permanent member. A pair of singles from the Burned and Buried sessions, Woman Killing Man and Erl King, were also released in 1997.

Not content to stop experimenting, two unusual projects were released in 1998. Hotel Opera, a solo album by Foljahn under the name la Lengua Asesina, featured a more minimal sound; Train Songs, which featured all three members of the trio, dispensed with the lyrics altogether to allow instrumental tracks to grow and bloom in many different directions, highlighting the development and growing cohesiveness of this band. Weak Beats and Lame-Ass Rhymes followed, issued in 2000 by Smells Like Records. ~ Stacia Proefrock, Rovi