The Stephen Smith House

The Stephen Smith house has stood at 645 Lafayette Street for nearly 170 years. Maybe you’ve walked past it. Maybe you’ve read the plaque out front that offers a brief summary of Smith’s life. But you might not realize that he was a key figure in the pre-Civil War Underground Railroad.

The Stephen Smith House

Smith was born into slavery and bought his own freedom, and by the 1840s and ’50s had become one of the wealthiest black men in America, channeling his wealth into education programs, libraries, and schools for free blacks. He founded churches, including the Cape May AME Church on Franklin Street, and the first home for aged African Americans. He owned properties on Decatur, Mansion, and Jackson Streets, and others near Congress Hall.

Smith and William Whipper, his business partner and close friend, owned a lucrative lumber and coal business in Pennsylvania. When large groups arrived in Pennsylvania seeking freedom from the slave state of Maryland—too many to escape by wagon or on foot—Smith and Whipper devised a false end to their railroad cars to conceal them on the journey north.

Portrait of Stephen Smith by James Stidun, c. 1840

Smith was a target of racial attacks in Pennsylvania that eventually sent him to Philadelphia and later to Cape May, though it was not immune, since the resort attracted abolitionists and southern planters alike. Still, Smith constructed his retreat, and he also opened a hotel next door that offered $8 beach vacations to African Americans from Philadelphia. He died in 1873.

The Stephen Smith house was saved from the wrecking ball in the 1960s, when it had been slated for demolition under Cape May City’s urban renewal plan. Today, the house is owned by four siblings, the Hamptons, whose grandparents purchased the home in 1932 from Smith’s grandniece. Through grants and donations, they hope to restore the building, which will display artwork, furniture and historic artifacts original to the home—a testament to Smith’s legacy.

For more information on Stephen Smith and to learn about ongoing preservation efforts, visit

Historic information for this post taken from Cape May Magazine‘s Fall 2015 article Stephen Smith: Cape May’s Underground Railroad Leader by Barbara Dreyfuss.