The Ordnance Survey Name Books record William Middleton as the occupier of Elm Hill House in 1839 and mention the surroundings as "a very pretty demesne situated in a beautiful locality". At the time of Griffith's Valuation, William Middleton was leasing this property from the Poe estate when the house was valued at almost £8. The original house is no longer extant and a modern house has been constructed at the site.