Spencer is an unincorporated community in Henry County, Virginia, United States. It takes its name from its earliest settler, James Spencer Sr., who moved from Loudoun County to Henry County with his sons in the eighteenth century. Spencer's son ensign James Spencer, Jr. died of wounds suffered during the Revolutionary War. (On his death, his widow remarried Nathaniel Bassett.)

Spencer was the founding site of the Spencer Bros. Tobacco Company, as well as the D.H. Spencer & Sons Tobacco, both begun by the Spencer family, with operations at Spencer, and later at Martinsville, Danville and elsewhere. The family-owned firm later became one of the nation's largest manufacturers of plug chewing tobacco with its well-known brand 'Calhoun' and others. The Spencer family built Grassdale Farm, their tobacco plantation, beginning in the eighteenth century. Grassdale, once called "The Homestead," is on the National Register of Historic Places.
The Spencer family also controlled the Danville & Western Railroad (later merged into the Southern Railway), which stopped in the town, as well as a small collection of other buildings, including a post office, doctor's house and other appurtenances. The family later sold their tobacco company to the R. J. Reynolds Tobacco Company in one of the first consolidations in the industry. The firms of D. H. Spencer and Sons and Spencer Brothers agreed in December 1903 to form a corporation with Reynolds in return for stock in the enterprise. R. J. Reynolds had grown up in nearby Critz, Virginia, and he and the Spencers were bitter rivals.
Grassdale Farm was once owned by Thomas Jefferson Penn, who built Chinqua-Penn Plantation outside Reidsville, North Carolina, where the Penn tobacco-manufacturing interests were located. The Spencer family and the Penn family are related (Jeff Penn's mother was Annie Spencer Penn, and the Spencer coat-of-arms appears above the entry at Chinqua-Penn.) 'Jeff' Penn sold Grassdale to his first cousin Margaret Dillard (née Spencer) Shackelford and her husband Dr. John Armstrong Shackelford, who subsequently restored the home.
In addition to Grassdale Farm, the Spencer-Penn School and Aurora are listed on the National Register of Historic Places.
Margaret Spencer Shackelford's sister Mary Holt married Kennon C. Whittle, a justice of the Virginia Supreme Court of Appeals, who lived at Belleview, built by their shared ancestor Major John Redd. A third sister, Blanche Spencer, married Julian H. Robertson Sr. of Salisbury, North Carolina, a textile company executive, private investor and philanthropist.
Spencer is part of the Martinsville Micropolitan Statistical Area.