On 2 March 1788 Captain Arthur Phillip sailed north to the inlet described by Captain James Cook in 1770 as a "broken land". Phillip explored the southern arm of Broken Bay and declared it "the finest piece of Water I ever saw". He "honoured [it] with the name of Pitt Water", after William Pitt, the Younger, who was Prime Minister of England.

Initially, the area was beyond the law, harbouring escaped convicts and smuggled rum. These convicts lived in caves or rough shacks and attempted to survive as best they could in the bushland. In 1819, a constable was appointed to bring the rule of law to Pittwater. In 1843 a customs house was built at Barrenjoey in an attempt to limit rum smuggling.