What Do You Think: Revamped Course Rating

[Chip Gaskins]

I realize many of you loathe the whole golf course ranking system and say that it is simply too hard to rank something as random and esoteric as a golf course, BUT.......we do......and I would like to know what you guys think is the fairest ranking system out there. 

Note, I realize they are ALL flawed, but what are the key components that make a good one.

Do you think there is any think we can learn from what Robert Parker has done with The Wine Advocate Or the Michelin "Star" rankings for restaurants that we could apply to a new/better golf course ranting process? Or the Zagats system? Or Business Week graduate school ranking

1) Golf Digest: Ambiance, Memorability, Conditioning, Shot Value, Playability, Design Variety, Aesthetics, and Resistance to Scoring

2) Golf Magazine: "There are no set-in-stone criteria our rankers must follow. Certainly, nearly all consider how a course tests the full range of skills, design rhythm, variety, setting and conditioning, among others. However, the weighting of these is left to the individual preferences of our panelists."  "use decimals"

3) Golf Week: Good Question?

4) Doak Scale: We all know it

5) Top100GolfCourses.Co.UK:

   1. Number of times that a golf course has appeared on a ranking table
   2. Weighted in favor of courses appearing the World rankings
   3. Weighting course appearing most recently in any ranking tables
   4. Check and balance by our own team of passionate aficionados
   5. Then open to website visitors who have played (hopefully) a course to rank it 1one to six balls.

Again, what are the key components that make a good ranking system

Should we just do:
Top 5, then go 10-20, 20-30, in no specific order, etc
or
Top 25, 25-50, in now specific order, etc

[John Kirk]

Chip,

Golfweek has a comprehensive set of rating criteria.  This week's issue should reiterate how the courses are rated, but when it comes down to it, the courses are rated 1-10, where a 10 indicates the rater thinks it's a top 5 course, 9 means it's top 15 (or something like that), and so on until a 5 means it's top 500, or something like that.

[Sean_A]

Chip

Personally, I prefer to allow the raters to use whatever criteria they for evaluating a course.  It all comes down to preferences and we all can justify this, that and the other - the question is to get a decent cross section of panelists which represent all aspects of course evaluation and perspectives.  I don't like the idea of assigning a ranking number to a course.  I would rather see rankings done in clusters of courses which are more or less equal, a cluster could be 1 or it could 50.  I would then like to see the clusters separated by a clear division - say the difference between getting 50 votes compared to 40 votes.  Trying to decide ranking numbers of course is crazy. 

For instance, I may cluster the following using any criteria I like which basically tends to be where would I be happy to pay full whack for a revisit based on a lot of factors some of which I clearly have nothing to do with golf and focus on the experience (but then I think everybody is influenced by outside factors - its just a matter of who admits as such) and only mentioning those in which I have a decent memory of the course:


A. Enniscrone, Carne, Pennard, St Enodoc & North Berwick

CLEAR DIVIDE

B. Portstewart, Ballybunion, Sandwich, Prestwick, Brora, Donegal, Tenby, Burnham, Portrush, Sligo, Southerndown, Harlech, Aberdovey, Beau Desert, Addington, Woking, Huntercombe, Whittington Heath & Kington.

CLEAR DIVIDE

C. Perranporth, Lahinch, Dornoch, The Island, Saunton East, Machrihanish, Castletown, Southerness, Princes, Dunbar, Deal, Cruden Bay, Aberdeen, Littlestone, Conwy, Co. Down, Portmarnock, TOC, Nairn, Wallasey, Saunton West, Westward Ho!, Gullane #3, Woodhall, Stoneham, Little Aston, Delamere Forest, Notts, Worplesdon, North Hants & Handsworth.


All of these courses are very good to great and I would like to see them all again, but I don't see how I can clearly rank them from 1 to whatever.  Does this make sense?

So far as any system that already exists the top100 ranking is far and away the best at least for the UK & Ireland.  Of course, this system can't exist without the others so as a stand alone system it can't work. 

Ciao

[Rich Goodale]

Chip

I've advocated and elaborated on the beneifts of a Michelin type system on this board many times over the past 7 years.  I'm too lazy/busy to use the search function today, but it's out there in this small GCA.com portion of cyberspace.

Rich

PS--if you think about closely, the 10 point Doak system is really a Michelin system in disguise.  Just forget about all the 1-6 Doak ratings and you essentially have the 1*, 2** and 3** courses (Doak 8-10).  From those 1-6's you can deduce from Tom's prose which are worth visiting (i.e. the 90+% of Michelin restaurants restaurants which are not starred but are recommended).

RFG

[Mark Bourgeois]

I say a closed or parametric system is needed given how everybody seems to believe the value / action is just in the top 50 or at least 100.

Pareto time!

Please no requests for clarification / retraction on any of the above: too lazy / busy....

[Rich Goodale]

Mark

I'm always too lazy to spend any time on the NEW!!  IMPROVED!! but still crap GCA.com "Search" function, and too busy to do anything more than say I am "too busy."

Except of course when the topic of rating sytems comes up....

I remember well my two years in the US Army, when much of my job involved the use of Office Efficiency Reports to rank people for the purposes of promotion and high-level training, e.g. War College (oxymoron or unintended irony?--you decide).  At that time, all officers were ranked on a 2400 point system (0.0 to 240.0).  This was based on averaging all of the OER's for their career, each of which was graded using 48 questions on a 0-5 point scale.  Anybody with an average of below 235.0 would never get past captain, unless their father was 4**** officer.  235-236 meant you made it to Major, and 237-8 meant Lieutenant Colonel.  To get to Colonel you needed at Least 239.0, and to make it to General, anything below 239.8 scrubbed you.

Why is this relevant?  I don't know and/or am too lazy/busy to care....

Slainte mhor

Rich

[JohnH]

I've always felt pulling names out of a hat worked well....

[Mark Bourgeois]

Bon Vivantism just took a major step backwards with that post.

I rate it 1* for Bon Vivantism, but 239.8 for conditioning!

[Rich Goodale]

Mark

We all have dirty little secrets in our past (or even our present) but only a few of us have the cojones to face up to the fact.

Defianty

Ricardo

[Chip Gaskins]

i did honestly try about twenty different searches to find old threads and never found anything other than architect interviews.  i tried "rating scale", "ranking scale", "doak scale", "michelin scale", etc.  i obvious haven't figured out the search feature yet.

[JESII]

When you sit for your first drink after the round you have to fill out a very brief (one question) questionaire...

How many holes do you remember?

15 or more is in the top block
12 - 14 next
9 - 11 next
etc...


I think block type rankings make the most sense because arguing #25 versus #26 is about the most useless thing in the world...I think this is where Tom Doak's deal makes sense. It has to be about INTERESTING GOLF.

[Chip Gaskins]

would we all advocate a #1 or #1 and #2 course?

it does seem Cypress and Pine Valley have established themselves in a little different category at the top of the Top 10, no?  at least if you look at the Golf Magazine numbers, those to course are far above the #3 course.

[hhuffines]

Chip,

I like the Doak scale myself, especially pertaining to the question of travel time and expense to get to a particular course.  I also think GOLF does a decent job at attempting a world top 100 ranking. 

The NC golf panel will be coming out with their NC list and it usually has a few questionable rankings...

[John Gosselin]

When you sit for your first drink after the round you have to fill out a very brief (one question) questionaire...

How many holes do you remember?

15 or more is in the top block
12 - 14 next
9 - 11 next
etc...


I think block type rankings make the most sense because arguing #25 versus #26 is about the most useless thing in the world...I think this is where Tom Doak's deal makes sense. It has to be about INTERESTING GOLF.

People remember the really bad holes just as much or more then the good ones. You might need to define "how many hole do you remember".

[Tom_Doak]

Chip:

I think the worst aspect of the top 100 lists is ranking them in order.  That's what causes the 33rd best course in the world to think it's somehow going downhill because it was #31 last time.

Sadly, I have to take some responsibility for that.  In the early days of these rankings, there were no numbers involved at all ... the final lists were arranged in groups of ten, decided by the editors based on panel input.  When I started compiling the top 100 list for GOLF Magazine many years ago, I made the panelists vote on all of the courses nominated, so there were numbers for each.  When I tabulated the results, I asked George Peper if he still wanted them in groups of ten or in order -- and he was surprised to hear that we could put them in order, and have a #1 course.  It generated much more interest in the rankings, so GOLF DIGEST and others followed suit immediately.  But I'm not sure it's been good for golf architecture.

At that time, I thought it was silly that you didn't know which course was 3rd, which was 10th, and which was 11th ... because a "top ten" is an artificial construct and the courses within each group aren't necessarily equal.  The advantage of the Doak scale is that you don't have to downgrade Merion or Ballybunion to make room for Sand Hills, and everything that's a ten or a nine is on an even plane.

P.S.  Rich is correct -- the top end of the Doak scale is sort of like the Michelin scale, although I suspect that a lot of the 7's would be a one-star establishment on any commercial list.

[JESII]

People remember the really bad holes just as much or more then the good ones. You might need to define "how many hole do you remember".

Of course...how about just by adding the word "fondly" at the end of the sentence?

Regardless, I agree that numerical rankings are unnecessary, even at the very top...one would have a difficult time convincing me that any course is unquestionably better than Shinnecock, but I'd put Pine Valley right with it. Have not played Cypress Point. Merion might be a notch below, but only a small notch...and on and on...

I loved the "Top 10" and "Second 10" type lists, but I do think I would prefer the 5 star, 4 star, 3 star type of comparison.

[Rich Goodale]

i did honestly try about twenty different searches to find old threads and never found anything other than architect interviews.  i tried "rating scale", "ranking scale", "doak scale", "michelin scale", etc.  i obvious haven't figured out the search feature yet.

Ok, Chip, I'm not as computer-illiterate as I sometimes think.   From my 2 minutes of searching, try this one on for size.

http://golfclubatlas.com/forum/index.php/topic,11996.0.html

Rich

[Tom Huckaby]

I loved that old thread. 

Still not sure I like the Michelin system for golf courses, though... but I speak as a consumer, one who likes to split hairs, one who wants to know out of each star category which is seen as better, and if so, why.  That is a lot to ask though...

TH

[Matt_Ward]

Just to echo what Tom D and Rich G have mentioned -- I am a firm proponent of cluster rating -- that is linking 8-10 courses in comparable qualities.

The silly idea that you can with any degree of certainity and say course "X" is now #46 rather than #48 or #49 is really inane.

Placing courses in some sort of "bunch" works well because it allows specific different styles to be recognized in such groupings.

One final point -- I do like the aspect where public courses are rated to clear dollar amounts -- for example, the best public under $100 would be most helpful. It itemizes architectural elements while at the same time realizing that not everyone will ever get on the super elite courses that are more private than the Vatican.

[Chip Gaskins]

Interesting post from 2001 that cut and paste from Guest: ForkaB

I like this, but 10 pretty detailed aspects to measure could be a little over the top and lead to even more confusion / subjectivity

"Great Golf Course Architecture

Criteria

1.   Wholeness.  A “course.”  This is to say something that flows, naturally and seamlessly, over the land.  Tee to green to tee to green, etc.
2.   WYSIWYG.  What you see is what you get.  Holes which, even if partially blind, make it clear, from their topography and the placement of hazards and the green, what the risks and rewards are of shots of varying intensity across varying angles.
3.   Subtle deceptions.  Within the context of 2 above, the addition of little tricks which reward experience, require you to think, and punish you if you do not.
4.   Diversity.  A mixture of shot values required, in terms of clubs to be used and the shape of the shots to be executed with those clubs.
5.   Greens which accommodate both the aerial game and the ground game.
6.   Greens which give higher rewards to properly executed “aerial” shots, with commensurately higher risk.
7.   “Lines of charm” closely linked to hazards.  Rough which borders the “fast lane” off the tee.  Greenside bunkers which eat into the putting surface.
8.   A balance between the use of bunkers and swales and rough and natural watercourses as “hazards.”
9.   Greens which have at least 4 good-excellent pin positions, and which allow alternate ways of approaching possible pins.
10.   Gut feel. You can’t always define a 3*** course, but you know one when you see one.

3*** Courses meet all of these criteria, in substantial part or in full
2**   Courses meet most of the criteria in full, but have some deficiencies
1*     Courses meet some of the criteria in full, and some or all partially, but have significant deficiencies

[Tom Huckaby]

Matt:

I agree that in the end it is silly to say one course is #46 and another #48 and thus the former is superior.

BUT... I still want to know which course is seen as better.  I want to know if a system thinks Cypress Point is superior to NGLA (for example) and if so, why.

Thus in the end - as a consumer - I prefer the existing ratings to any clustered system.

I do realize however the damage this causes the golf business... but still, such damage wouldn't go away in a clustered system.

So what's the answer?  To not take these things so seriously, that's what.  But I know that has little chance of happening either....

[John Kavanaugh]

The secret to these lists is simply to get to know the raters and weigh the value of the rankings based on their personalities.  When I think Golfweek I think of Paul Thomas.  When Digest - Jim Franklin and Golf Magazine...Pete Dye.

[Matt_Ward]

Huck:

When you say you want to which course is "better" -- please -- think of the movies in which the gorgeous gal disrobed but your didn't see the whole figure. Leave a little to the imagination.

I remember something Tom Doak said and I am paraphrasing - but American has roughly 12 top ten courses. To say which one is "better" is really a commentary on particular preferences at any given moment. There is no 100% correct answer.

When you cluster courses you are recognizing their greatness AND their unique differences at the same time.

The fascination with who is #1 is more akin to college football polls and doesn't work at all with golf course assessments.

[Tom Huckaby]

Huck:

When you say you want to which course is "better" -- please -- think of the movies in which the gorgeous gal disrobed but your didn't see the whole figure. Leave a little to the imagination.

I remember something Tom Doak said and I am paraphrasing - but American has roughly 12 top ten courses. To say which one is "better" is really a commentary on particular preferences at any given moment. There is no 100% correct answer.

When you cluster courses you are recognizing their greatness AND their unique differences at the same time.

The fascination with who is #1 is more akin to college football polls and doesn't work at all with golf course assessments.

I see that as a cop-out; sorry, Matt.  I see that such a system recognizes their greatness, I just don't see how it also recognizes their unique differences.  How can it do the latter when it lumps so many as being essentially the same?

Hey, I know there are no easy answers here.  I still like mine the best:  don't take these so seriously.

But since people do, well.... I still remain interested in REALLY splitting hairs.  And I know that it's never going to be right or wrong; I just am more interested in that than copping out and not even trying.

TH

[Peter Pallotta]

I think the *number* of categories serve to create differences unnecessarily, and artificially.

If you judge Playability do you also need to judge Shot Values or Resistance to Scoring? If Aesthetics, also Ambiance? Memorability seems to me a *result*, not a criteria. Conditioning seems silly, it's so variable. 

How does it Play, How does it Look and How is it Designed seems enough, and maybe better.

Peter