In 1959 Herald joined the Greenbriar Boys with John Yellin and Eric Weissberg. While the band garnered a good rep playing Sunday jams in Washington Square (in Greenwich Village) and local American Youth Hostels, they had few commercial aspirations early on. This outlook changed, however, when Yellin was replaced by Paul Prestopino and then Ralph Rinzler. Rinzler encouraged the band to practice more often, and by 1960 the group traveled to Union Grove, North Carolina where they won first prize in the band competition (the first Northern band to do so). In 1962, the Greenbriar Boys guested on Pal of Mine and Banks of the Ohio on Joan Baez's second album. From here it was a short step for Vangaurd's Maynard Solomon to sign the group. After performing on the compilation, New Folks, the group recorded three successive albums for the label, The Greenbriar Boys (1962), Ragged But Right! (1964), and Better Late Than Never (1966), and one album for Electra, Dian and the Greenbriar Boys (1963).
While holding down his guitar and vocal chores in the Greenbriar Boys, Herald went to work at the Smithsonian Institute and managed bluegrass legend Bill Monroe. Vanguard also kept him busy with session work during the '60s. He played an essential roll in the work of Ian Sylvia as they expanded their folk sound in the mid-'60s, and also played on albums by Jack Elliott and Doc Watson. In 1972, Herald recorded his first solo album for Paramount and followed with the electric John Herald and the John Herald Band in 1978. By the early '80s, however, he had returned to his acoustic roots. "… Most of the traditional folk music people that I know …," Herald noted, "will always be playing only at home if need be, like we did when we first started." ~ Ronnie D. Lankford, Jr., Rovi