The land originally known as Kellys Swamp had been impounded for hundreds of years by beaver colonies.
The stream was named after Captain William Hill who settled in the area around 1820.
At one time, a small mine was located near the present-day swimming beach. The mine extracted pigment for the paint industry.
The park opened in 1953 and is named for the creek that runs through it.