This house was built in stages. The oldest part was probably built by Benjamin Birkett Parlor about 1857. Parlor was appointed clerk of the newly formed District Council of Rapid Bay on 15 September 1856, but with a large family to support he got into financial difficulties and embezzled money from the Council. Parlor was tried in 1874 and sentenced to 18 months imprisonment. After serving his sentence he returned to the Rapid Bay district for a while, but subsequently moved to Warooka.

The property was acquired by William Whyte in 1874 (Whyte also succeeded Parlor as Council clerk), but he sold it in 1876 to Henry Charles Swan, one of the owners of Bullaparinga Station. It was probably occupied a short time by Francis Warwick, the manager of the station. During this period the house was substantially enlarged. After Bullaparinga Station was sold, this house was bought back by William Whyte, who lived in it from 1882 until his death in 1912.