One of the Cape Fear's most exclusive communities lies on an island which didn't begin to see permanent development until the second half of the 20th century.