Spencer is an unincorporated community in Henry County, Virginia, United States. It takes its name from its earliest settler, James Spencer Sr., who moved from Loudoun County to Henry County with his sons in the eighteenth century. Spencer's son ensign James Spencer, Jr. died of wounds suffered during the Revolutionary War. Spencer was the founding site of the Spencer Bros. Tobacco Company, as well as the D.H.