During that time, a resort hotel with grand ballroom was developed on the island and it became a place for the beautiful people to be seen. I don't remember the history well, but the developers of the resort, probably the aforementioned brothers, became very involved in the growth of the city and in attracting the upscale clientele. Apparently there is some good fishing, sailing, and diving around the islands. I suspect that the cooler weather, proximity, and more relaxed environment were factors for those seeking to escape the summer heat in L.A.
Regarding the golf, a thing to remember is that back in those days, the quaility of the course was much less important to the pros than the purse, inducements, and amenities offered.