The Brown House was built in 1853 by the H.W. Winslow Company as a boarding house for it's saw mill employees. This historic building became the home of "Grandfather Brown" and his sons in 1868 when the sawmill and properties were bought by W.W. Brown from the H. W. Winslow Company. Today, it is the headquarters of the Northern Forest Heritage Park and house the offices of the Androscoggin Valley Chamber of Commerce, United Way of Northern NH and Androscoggin Valley Economic Recovery Corporation.
In 1929, Mrs. O. B. Brown had a formal garden installed on the lawn between the house and the church next door. The garden is currently being restored by the Heritage Garden Club.