Built in the 1780s by Joseph Massey Harvey, a Quaker merchant, who married Rebecca Fisher and still occupied by him in 1814. Joseph and Rebecca were the parents of William Henry Harvey, a distinguished botanist. At the time of Griffith's Valuation Summerville was valued at £33 and occupied by James T. Fisher who held the property from Eliza Harvey. The house is now an accommodation and conference centre of the Mary Immaculate College for training primary teachers. The college was founded in 1898.