"The purpose of this Society shall be to bring together those people interested in history and especially the history of Pound Ridge and environs. Understanding the history of our community is basic to our democratic way of life, gives us a better understanding of our state and nation, and promotes a better appreciation of our American heritage."

The building was erected in 1853 by the Presbyterian Church of Pound Ridge to serve as a lecture room. In 1921, the building was sold to the town of Pound Ridge for use as a Town Hall. The building was leased to the Historical Society in 1982.