Talley was raised by his beloved grandmother in the Jim Crow South; much of his first memoir is dedicated to his most cherished memories of his time with her in North Carolina.
After graduating from North Carolina Central University, he was awarded a scholarship to Brown University, where his time as a post-graduate student in French would lead to a move to New York City and build a legendary career in fashion. He remained a devoted member of Harlem's Abyssinian Baptist Church and a tireless board member at SCAD University, where he helped to stage blockbuster exhibitions.