Not only was Hong Kong hardly a “barren island” when the British arrived in the late 1830s, but Hong Kong’s people were far from the “handful of fishermen and pirates” they have often been described as. As historian and former colonial official James Hayes explains, farming was the “principle occupation.” The island had “several villages of some size, as well as hamlets, and a few larger coastal villages which served as market towns for the villages and as home ports for a permanent boat population and visiting craft.”
Hayes concludes that “long before 1841,” the inhabitants of Hong Kong Island had “settled into the routine of a settled life. Tied to their fields and houses, and to their businesses and daily occupations, they had established institutions of the kind that is usual in Chinese communities, including the shrines and temples that were the object of periodic and special rites through the calendar year.” Hong Kong’s temples alone, argues Hayes, prove that “the island was certainly well‐established in settled communities long before 1841.”
— A Concise History of Hong Kong
See City on the Edge: Hong Kong under Chinese Rule, chapter 2, for more evidence.