Critics, Tastes, Rankings--restaurants vs. golf courses

[Craig Sweet]

Golf course ratings should go the way of YELP....yelp.com....the only place to go for honest restaurant reviews!

[Adam Clayman]

The anomaly that comes to my mind is when the Scots and Brits come to sunny Florida and much prefer that golf to their own.

Is this some sort of grass greener rating system?

[Tom Huckaby]

In the end I just wonder if the positive of anonymity would outweigh the negative of these assessments becoming that of a small bunch of what would have to be very hoity-toity people.... We'd know how THEY felt about golf courses... but man I wonder if I really want the opinions of guys like that - at least IN TOTALITY...

I can't say I know much about how golf rating is done (there HAVE been countless threads on the issue, but I guess I just wasn't paying attention), but my understanding would be that those raters have to have the wherewithal to travel a lot, and play a lot of golf, no? Isn't that travel at their own expense? On some level, aren't the CURRENT rating systems being performed by a relatively small group of fairly hoity toity people?

To some extent, yes.  But it depends on the panel, and the panelist.  I can tell you I nearly never travel expressly for the purpose of rating a course; I fit that into my travel otherwise planned.  And I'd have to guess that at least for Golf Digest, I am the norm.  Nevertheless, you do make a good point that we raters necessarily do have to be at least a little hoity-toity.

But compare it to a system where perfect anoymity is maintained, and hopefully you get my point. 

Call the current guys Hoity-Toity 4s on the 1-10 scale... under an anonymous system they'd all have to be 9s or 10s.

TH

[Mark Bourgeois]

JC

This is exactly my point. If someone lacks the moxie and position to access these courses, then that person lacks the minimum qualifications to judge, particularly given they pay their own travel costs.

A further bonus is that those who can prove they are qualified likely would have to show they actually have a broad and deep golfing experience. It is a self correcting mechanism and makes a lot more sense than having all these training sessions to "educate" raters (the very fact that someone must be trained shows how poorly qualified many are) or changing the criteria (ditto).

Mark

[Tom Huckaby]

Mark:

On all the rating panels now, there is a vetting process.  All panelists do have to "show they actually have a broad and deep golfing experience."  Now of course it is not perfect, and bad apples do get by.  If Kavanaugh were here he'd say I was one of them.   

But I tend to prefer a broad spectrum of opinions to those that would come from the tiny group who could indeed access ALL courses needed to be accessed, anonymously.  In fact it scares me just who could actually do that and remain completely anonymous.  I am honestly not sure I want the opinions of someone like that - at least not EXCLUSIVELY.

TH

[Sean_A]

Rich -

Good thread. I'm happy you've revived these long time issues.

Rating courses rasises a nest of diffucult issues that aren't thought about very much. To touch on one:

The idea of precise mathematical ratings necessarily assumes that there are common denominators - a set of descriptions that are neutral across courses - on which you can make apples to apples comparisons. An idea, of course, which is preposterous. The absurdity jumps off the page. How do you assign a set of number values to features on courses as diverse as NGLA, Firestone, TPC Sawgrass and Garden City?

In the alternative, most rating systems these days are just voting mechanisms. Given that there are - essentially - no criteria for who votes (other than who you happen to know), there is no a priori reason why their voting outcomes ought to be given any weight whatsoever.

I'm not sure the Richelin system works either. Or at least not without some more thought.

The Michelin restaurant stars works because over the years a very limited number of food raters have gained credibility. They have earned our acknowedgement of their expertise. But more important is that they are motivated to pick the right restaurants because Michelin is motivated to get it right. On pain of losing its credibility as a guide, and thus as something you or I might buy.

So there are factors in place that causes Michelin to take great care with their picks. They have a direct economic stake in getting it right. They have a downside if they get it wrong. So they pick carefullly, conservatively and with great deliberation.

I know of no entity that sponsors the rating of golf courses that operates under similar, healthy constraints. And until they do, golf rating will always be suspect. Until they do, golf ratings ought to be seen as a lark that no one should take very seriously.

Which is where I find myself on the issue.

Bob      

Bob

In a way you are right in that the Rihcelin Guida or any system for golf doesn't have street value as it were.  These are throw away lists that can be had for free.  However, so far as I know, the raters for Michelin aren't really known.  What we trust is the system, not particular raters.  This same sort of trust hasn't been built into golf rating systems.  Perhaps if the system was more like Michelin and it cost money to purchase the guide than more respect would be forthcoming.  I don't know.  What I do know is that concepts like value for money (which despite its reputation for class, Michelin does quite well in sorting out) and anonymity are major oversights imo.  I know that anonymity is hard to control, but not every rater needs to see all the top courses to know its a top course - if you know what I mean.

Ciao

Ciao

[JMEvensky]

Rich -

Good thread. I'm happy you've revived these long time issues.

Rating courses rasises a nest of diffucult issues that aren't thought about very much. To touch on one:

The idea of precise mathematical ratings necessarily assumes that there are common denominators - a set of descriptions that are neutral across courses - on which you can make apples to apples comparisons. An idea, of course, which is preposterous. The absurdity jumps off the page. How do you assign a set of number values to features on courses as diverse as NGLA, Firestone, TPC Sawgrass and Garden City?

In the alternative, most rating systems these days are just voting mechanisms. Given that there are - essentially - no criteria for who votes (other than who you happen to know), there is no a priori reason why their voting outcomes ought to be given any weight whatsoever.

I'm not sure the Richelin system works either. Or at least not without some more thought.

The Michelin restaurant stars works because over the years a very limited number of food raters have gained credibility. They have earned our acknowedgement of their expertise. But more important is that they are motivated to pick the right restaurants because Michelin is motivated to get it right. On pain of losing its credibility as a guide, and thus as something you or I might buy.

So there are factors in place that causes Michelin to take great care with their picks. They have a direct economic stake in getting it right. They have a downside if they get it wrong. So they pick carefullly, conservatively and with great deliberation.

I know of no entity that sponsors the rating of golf courses that operates under similar, healthy constraints. And until they do, golf rating will always be suspect. Until they do, golf ratings ought to be seen as a lark that no one should take very seriously.

Which is where I find myself on the issue.

Bob      

I,too,agree with the last couple of paragraphs.

I presume those who rate for Michelin have some "professional" knowledge of food as opposed to just being people who like to eat.Is there the foodie equivalent of an 18 handicap rating restaurants for Michelin?

Wouldn't golf course ratings be more "valid" if the raters were somehow more "professional"?

[BCrosby]

Sean -

The anonymity of the raters is a good thing, but it is much less important than the fact that there is a downside to getting it wrong. That's the critical distinguishing feature. There is no real downside to golf ratings getting it wrong.

Partly that's a structural thing. I buy Michelin or other guides because I can choose to eat at any restaurant in Paris. So they sell a lot of such guides. It's a big business with lots of competition.

As for golf courses, I can't play every golf course around Philly (for access reasons), so I'm not going to pay up for a good guide to the quality of all courses in the area. Given my universe of choices, it's a largely irrelevant metric. It's not something I need to know.

Bob  

[Mark Bourgeois]

Mark:

On all the rating panels now, there is a vetting process.  All panelists do have to "show they actually have a broad and deep golfing experience."  Now of course it is not perfect, and bad apples do get by.  If Kavanaugh were here he'd say I was one of them.   

But I tend to prefer a broad spectrum of opinions to those that would come from the tiny group who could indeed access ALL courses needed to be accessed, anonymously.  In fact it scares me just who could actually do that and remain completely anonymous.  I am honestly not sure I want the opinions of someone like that - at least not EXCLUSIVELY.

TH

Tom

Regarding "ALL," would the "accessible" list of courses really be that different under an anonymous panel?  Do raters just walk up to the counter at Cypress, Shinnecock, National, or ANGC and present their card?

And anyway, it's a little frustrating that people have attached to the anonymity comment.  As far as accessibility goes, the far more damning statement is the "one and done" approach to rating. Under this current system of publicly-known raters, they still blow in for just one round in the vast majority of cases.  I could see where accessibility might have some value if a rater was required to visit the course a minimum of 4 times and / or under a stated variety of conditions.

Mark

[Sean_A]

Sean -

The anonymity of the raters is a good thing, but it is much less important than the fact that there is a downside to getting it wrong. That's the critical distinguishing feature. There is no real downside to golf ratings getting it wrong.

Partly that's a structural thing. I buy Michelin or other guides because I can choose to eat at any restaurant in Paris. So they sell a lot of such guides. It's a big business with lots of competition.

As for golf courses, I can't play every golf course around Philly (for access reasons), so I'm not going to pay up for a good guide to the quality of all courses in the area. Given my universe of choices, it's a largely irrelevant metric. It's not something I need to know.

Bob  

Bob

Thats a good point.  I wonder how limited the market for a properly researched/conducted golf guide is?  Especially given all the free stuff about and the many classic and not so classic coffee table books on the market.  I think I would buy it because I would be interested in knowing what a system I trust has to say - even though I am not gonna play loads of those courses. Its just helps me narrow my choices if the day ever comes that I have a wide choice.

Ciao

[Tom Huckaby]

Mark:

OK, are we talking perfect world, or are we grounding this in some sort of reality?

Because I fully agree that in a perfect world, all courses should be rated anonymously, and each should be seen a minimum of 4 times, preferably one in each season and/or possible weather situation.

But that has ZERO chance of happening.

So let's set out terms here before we go any farther.  Do you want to talk perfect/dream-world, or do you want to talk what in reality could possibly be done?

My takes are grounded in the latter.

TH

[Peter Pallotta]

Rich -  I'm a fish out of water here. (Fry me in butter, squeeze on some lemon, and serve me with riesling).

I wish I was able to rank courses.  I have no idea how it's done.  I can tell you the course I like most and the course I like least, but I have no clue what my 9th favourite course is or how it compares to my 27th favourite.  And trying to rank courses 14, 15, and 16 would make me mental...

I understand the idea of not "comparing" courses, and of instead saying that this course is a 10 and that one a 7.  But placing in comparative order the courses I've played I find impossible, and I think that even the 10s, 9s, 8s scale has an element of this comparison in it, at least implicitely

Peter

[Mark Bourgeois]

Peter

Excellent point.  A person who knows it can't really be done is much preferred to someone convinced it can.  You're on the panel!

Tom, absolutely a workable system can be devised.  But first the concept, then the application.  True or false:

Panelists demand that architects spend a lot of time on site getting the routing/detail work just right. And yet, what do many panelists then do? Blow in for a quick afternoon round and then off they go. They don’t see the course in different winds, different seasons, different playing conditions or get to experience different hole locations. They make a snap judgment based on one visit. As a course owner, you can only hope that they played well.

Mark

[Tom Huckaby]

Mark:

I think we got off on a wrong foot here.  I do not intend this to be contentious; nor am I defending the current system.  I concur that the process is odd, and the results at times are even odder.  BUT... to correct one thing, I don't think panelists demand anything in terms of what a course must be... we just assess as per the rules of which magazine we do it for. And it's very unfair to say we also all just blow in and base things on how we played.  So, FALSE.

But yes, the current system does suck overall.  It silly to think one course is better at #55 than the one at #72.

But I also am at a loss as to how your suggested way could be implemented, not in the real world.  I do not see a real-world way that a system of perfect anonymity and multiple visits could be achieved.  There are too many courses and that means way too much cost.

The only way would be a group of people doing this full-time, either independently wealthy and willing to devote 100% of their time to the efffort, or financed by the magazines (or other sponsoring entity).

I don't see that as having any chance of happening.

Do you?  And if you do not, how else could this be achieved?

TH

[Mark Bourgeois]

For starters, one's opinion could be discounted according to how many times they'd seen the course within a time period.  For example, count only third-look reviews.  Or third views count full, second just 67 percent, and first just 33 percent. Or a simple bonus system could be put in place where the first look was at standard or par, then each additional review (provided within a certain time frame plus the course had not been substantially modified) counted an additional 1-2 percent, up to a max of, say, 10-15 percent.

So the first time Bobby Jones saw TOC it would not count nearly as much as the 10th time, meaning that his early views, which by his admission were wrong, would not have damaged the credibility of the rankings.

Similarly, Tom Doak's opinion of TOC would count more than many others', given the time and effort he put in there studying the course and serving as a caddie.

As far as anonymity goes, just have fewer panelists and / or review fewer courses. So what?  It would eliminate all the "random walk" B.S. you get in these lists, anyway, like Pebble rising up and down, which appear to serve no purpose other than to manufacture "news".

Mark

[Tom Huckaby]

Mark:

That's good stuff, but you really didn't answer my question.

I fully agree that your way would produce much better results.  Stipulated, agreed.  No more need to discuss.

I still cannot see how it could be implemented.  There are not enough raters, nor time, and there are too many courses to be seen.  You have not addressed that issue; that is, unless you do expect this to be done full-time; either by the independently wealthy who could devote themselves to the project, or those otherwise paid to do this.  In the first scenario, well I have stated my objection - I am not sure I'd care much for the assessments of such people, at least not exclusively.  The second I see having zero chance of happening and even if it did you'd have pressure to be beholden to advertisers - not a good thing.

So how will this really work?  I still can't see it.

TH

[Mark Bourgeois]

Tom

We are discussing two things.  1) The allocation of reviews (which approach to take: max out on # courses but see only once vs. multiple looks); 2) the absolute or total number of reviews conducted (possible anonymity issue).

Your hypothesis is too few (private) courses will be seen enough to receive a ranking due to multiple-looks requirement plus fewer total reviewers available under cloak approach.

Solution: extend the cycle. These lists in the best of times shouldn't come out more than once every 4-5 years, at least if the purpose really is to provide an accurate list.

It would be interesting to see if reviews went anonymous whether #2 really arose as an issue.  In the example, say, of Golf Digest, surely a network of 500 or so low-handicappers could get almost anyone in almost anywhere they now play today.

Mark

[Tom Huckaby]

Mark:

Well my hypothesis does not limit this to private courses; remember all the public courses need to be seen.  And while anonymity is not an issue there, expense is.  It also adds to the quantity of courses that need to be seen.

And extending the cycle could achieve this.  However, I see that as more fantasyland as magazines will never go for that - the annual or semi-annual listings are the coin of the realm.  Of course a non-traditional magazine could take this on... but then you get to the expense issues.

Sorry Mark, I still don't see away this would work in the real world, as great as it would be.

TH

[Mark Bourgeois]

Tom

It sounds like your position is that every course must be seen. Surely that's necessary just for the only legitimate ratings that are done, the state association slope / ratings.

And if not, extending the cycle solves the problem anyway -- as does the new-normal in course construction.

Besides, sure the regular magazines could extend the cycle, for two reasons:
1. For a general mag (GD, GM), ratings aren't that important commercially;
2. They could recut the data to manufacture all sorts of annual lists, saving the "greatest" for once every five years.  (Golf Digest has done this to a bewildering degree already, GM to a lesser extent. But both have done it.)  By spending more time on each course, raters would gain a greater appreciation for the various types of merit for courses (fun, difficulty, quirk, plus non-architectural stuff) and for individual holes.  They could produce *more* lists off an in-depth approach.  In a five-year cycle, it could go: most fun, toughest, beautiful / glorious, play-once-in-a-lifetime.  That's for courses.  And / or they could lard in lists based on holes: par 3s, par 4s, par 5s, and / or risk-reward, toughest, stunning, etc.

Mark

[Tom Huckaby]

Mark:

You don't think every course need be seen?  You think ratings should be given based on TV, pictures, something else?

As for the rest, you have sold me.

I still think you're gonna have issues getting anonymous coverage of all the courses that need to be seen - new ones in particular - but by extending the term, it might work and it might also nullify my "too hoity toity" issue.

I just still don't think it has a snowball's chance in hell of being implemented - there remains no reason for any magazine to do this, hell by their take the rating process works wonderfully - but you have sold me on how it COULD be done. 
TH

ps - thanks for your mention of the "legitimate" ratings; not sure if you know, but I do those also.  Of course a certain poster attacked me for doing those too... but we can discuss this in peace in his absence....

[Mark_Rowlinson]

Going back to the Guide Michelin, on the whole its starred restaurants are very expensive and the food is showy - a layer of foie gras covered in a jus de something impossible to obtain anywhere else in the world balanced on grouse stuffed with Beluga caviare held up on six frogs' legs surrounding a bed of truffled scallops cooked in vintage Krug. That's just one course. It is in the area of bibs that Michelin really comes into its element, identifying those good value retsaurants which still do peasant food, a rapidly declining species. However, take yourself to La Rochelle (or just about anywhere on the north and west coasts) and you'll almost certainly choose to eat in any of the plain, harbourside fish restaurants where the oysters, mussels, praires, etc come as fresh as can be accompanied by a bottle of Gros Plant or white from the Il de Re. No Michelin inspector will ever go near those restaurants - they don't do anything fancy, yet it is the gastronomic equivalent of unmodernised links golf (possibly on Machair). Get Michelin involved and the food gets fancy - and we know all too well what the equivalent golf courses are like.

[Mark Bourgeois]

Well, every course does need to be seen -- but everybody doesn't need to see every course.

There are statistical methods that could be used, relatively simple methods, to determine whether one reviewer's first look merits management sending them back for another and / or sending others to look at it, too.

Specifically:
1. The historical accuracy of the reviewer (adjusted for a few things like the fact that it's a first-time look)
2. Baysian analysis (basically, a way to guess how likely the course is to make some sort of list some where -- a first-year stats course is all anyone would need to apply the concept)

Anyway, word of mouth / post is really strong, which for intellectual honesty is a problem (Keynes's "beauty pageant" problem) but does ensure courses of potential significance get seen.  Nobody's missing Chambers Bay.

More importantly, if ratings really are important, then a multiple-look philosophy would do more than just make for a better list.  (Really, who cares about that?)  It would reward designers for getting the details right, for going beyond the dumb / accessible in design and giving us subtlety and sublimity, etc.

Mark

[Tom Huckaby]

Mark:

SOLD.  And I got that every course needs to be seen, but not by all raters.  That's what I assumed all along.

But I still don't think any of it has any chance of being implemented.  Not that I am happy about it, but I think my trump card which "wins" this argument is this:

no magazine has any incentive whatsoever to change how they do it now.  They get relatively free data, it works to give them rankings, and their readers and advertisers seem quite happy with how it all works.

But you are darn perceptive... care to take on what's in it for them to change?

[Mark Bourgeois]

Going back to the Guide Michelin, on the whole its starred restaurants are very expensive and the food is showy - a layer of foie gras covered in a jus de something impossible to obtain anywhere else in the world balanced on grouse stuffed with Beluga caviare held up on six frogs' legs surrounding a bed of truffled scallops cooked in vintage Krug. That's just one course. It is in the area of bibs that Michelin really comes into its element, identifying those good value retsaurants which still do peasant food, a rapidly declining species. However, take yourself to La Rochelle (or just about anywhere on the north and west coasts) and you'll almost certainly choose to eat in any of the plain, harbourside fish restaurants where the oysters, mussels, praires, etc come as fresh as can be accompanied by a bottle of Gros Plant or white from the Il de Re. No Michelin inspector will ever go near those restaurants - they don't do anything fancy, yet it is the gastronomic equivalent of unmodernised links golf (possibly on Machair). Get Michelin involved and the food gets fancy - and we know all too well what the equivalent golf courses are like.

Mark

The frogs probably won't go in those restaurants because of all the rosbifs!

I always assumed it was the reviewers who did the Bibs.  Are those done by slob / failed-out inspectors, a sort of GCA.com wing?

But the important thing is the contradiction in your last sentence: Michelin *does* offer recommendations for peasant food.  Not the formal tick-the-list reviews, but still they are recommendations.  In fact in just the past two weeks I ate at few starred restaurants as well as an even greater number of Bibs.  If Michelin wasn't involved, someone should inform them that forgeries of the red and green guides are being sold in respectable bookshops around the world!

Mark

[Mark_F]

Rich,

I think you are a little off beam here.

Three star restaurants are rated as such not because of how much better their food (architecture)is over two star establishments, but how much better their loos and cutlery (conditioning and number of bunker rakes) are.