"Charm" is interpreted in too many different ways. Sometimes to call something charming is to imply a diminishment - that the person or thing being described is a little smaller, a lesser version of that thing, but that in the lessening some kind of appeal is achieved. Am I making sense? This is, to me, different than when the word is used to describe the quality of a person (usually a man) who is deemed to be attractive because of some persuasive quality, the kind of guy who could "charm you out of your pants," so to speak. It's like there's some kind of lie there, in which both the charmer and the "charmee" are somehow complicit.
When I'm thinking about a charming golf course, I'm thinking that it can't be a big, brawny, penal, or impersonal-feeling course. It has to be just a little smaller, a little friendlier, whether by virtue of the course itself, it's surroundings, or the club that encompasses it. I don't know that Torrey Pines South, for instance, has much charm - Tiger's work there notwithstanding. But Torrey Pines North does have charm. Here in Denver there's a course like Murphy Creek, which I enjoy playing very much - but I don't see it as charming. It's just a big, fun golf course. But the old City Park muni, with it's crossing mounds and tilted greens, it is charming to me. To each our own, I guess.