He left Williams in 1949 to serve in the U.S. Navy, regularly performing at military dances. After returning to civilian life, Penny played behind Lefty Frizzell and Little Jimmy Dickens in addition to pursuing a career as a songwriter, and in 1954 Jean Shepard recorded his Don't Fall in Love With a Married Man. By 1955 Penny evolved from straight-ahead country into rockabilly, even traveling to Memphis to pitch his songs to Sun Records owner Sam Phillips. Earmarked for Sun sensation Elvis Presley, the songs were rejected and Penny relocated to Evansville, IN, where he regularly appeared on WEHT-TV's #Hoosier Hoedown and worked part-time as a disc jockey. In 1957 he was tapped to host WEHT's new #Hoosier Jamboree.
The following year, he and his band journeyed to Cincinnati's King Studios, recording the rockabilly cult classic Bip a Little, Bop a Lot for the Federal label. The record fared poorly on its original release, however, and in 1960 Penny returned to Florida, returning to radio as well as country music with the minor hit Frosty Window Pane. In the wake of a near-fatal auto accident he found religion, and in the years to follow focused on gospel music, most notably recording the LP Growing Old With God. Inducted into the Rockabilly Hall of Fame in 2001, two years later Penny was the subject of the NBM label release Large Economy Size, which paired unreleased demos from the 1950s alongside newly recorded material. In his twilight years, he also wrote -Lookin' Back on Hank, a memoir of his life on the road with Williams. ~ Jason Ankeny, Rovi