Try ranking courses the way GOLF DIGEST does

[Tommy Williamsen]

So I have an exercise for everyone. You are pretty good at belly aching about rankings and that GD panelists are unknowledgeable and lower than worm sweat. Why don't you rate courses you know well according to the formula GD uses and see how your courses fare? I have found that some courses I dislike immensely fare better than I would have thought and that some courses I love don't fare as well as I would have hoped. There are some state rankings that I would have numbered very differently if I did it on what I personally felt were the merits of each course. Yet, when I tried as honestly as I could to rate them, the order came out differently than I would have liked. I remember when I first play Bulle Rock. I disliked it----a lot. I wouldn't put it in the top twenty in MD. When I added up the scores it made my top ten.
You may argue that GD's formula is flawed. But that's not the point of this exercise. Be as honest as you can with states you know the best and see how they compare with GD's list and your own preferences. You might be surprised as to the results.
GD panelists play and score courses on seven criteria:
SHOT VALUES (double value)How well do the holes pose a variety of risks and rewards and equally test length, accuracy and finesse?
RESISTANCE TO SCORINGHow difficult, while still being fair, is the course for a scratch player from the back tees?
DESIGN VARIETYHow varied are the holes in differing lengths, configurations, hazard placements, green shapes and green contours?
MEMORABILITYHow well do the design features provide individuality to each hole yet a collective continuity to the entire 18?
AESTHETICSHow well do the scenic values of the course add to the pleasure of a round?
CONDITIONINGHow firm, fast and rolling were the fairways, how firm yet receptive were the greens and how true were the roll of putts on the day you played the course?
AMBIENCEHow well does the overall feel and atmosphere of the course reflect or uphold the traditional values of the game?To arrive at a course's final score, we total its averages in the seven categories, doubling Shot Values.

[Tom_Doak]

But it's your formula I disagree with, both in concept and in various specifics.  So why would I do any exercise that uses it?  You might as well ask me to go to your church a few times to see if it changes my mind about my religion.  (No offense intended; that was the metaphor I came up with before I realized whom I was addressing!)

[Steve Lang]

   what scale is to be used? 1-100? All relative to average or what??

[Jon Wiggett]

I really do not see the point of these type of lists. In the end it comes down to what do you think of the course.


Looking at each point individually


SHOT VALUES (double value)How well do the holes pose a variety of risks and rewards and equally test length, accuracy and finesse?

This is only really possible with multiple plays and indeed the higher a course would score on this point the more times you would need to play it in order to properly appreciate it.

RESISTANCE TO SCORING How difficult, while still being fair, is the course for a scratch player from the back tees?

How do you know this unless you are firstly a scratch player and secondly have played the course multiple times?

DESIGN VARIETY How varied are the holes in differing lengths, configurations, hazard placements, green shapes and green contours?

Quantifiable question IMO
MEMORABILITY How well do the design features provide individuality to each hole yet a collective continuity to the entire 18?

Quantifiable

AESTHETICS How well do the scenic values of the course add to the pleasure of a round?

Quantifiable

CONDITIONING How firm, fast and rolling were the fairways, how firm yet receptive were the greens and how true were the roll of putts on the day you played the course?

Surely the ideal is having a similar firmness of both fairway and green and this to reflect the design of the course. So, firm fairways and greens equals greens where a running shot is possible. Target style courses should be a little softer to accept a shot. Trueness of roll is always a case of what is meant by trueness. Putts that bobble a bit are not a problem as long as they bobble straight and at an even speed.

AMBIENCE How well does the overall feel and atmosphere of the course reflect or uphold the traditional values of the game?

Surely the last part of this should be uphold the concept of the facility.

I am not a great admirer of ranking that give scores but rather prefer opinions on courses where you find out about what the coure has to offer.

Jon

[Adam Lawrence]

'Firm yet receptive' is a classic case of trying to have your cake and eat it!

[Peter Pallotta]

Tommy -

that's one of the best questions I've seen here in a long time.

Speaking for myself, I simply couldn't/wouldn't know how to go about answering it, even with the 2 or 3 courses I know best and have played most often.

Some of that is a built-in bias of some kind, e.g. I tend to believe that if a course has very good and interesting/contoured greens it will have very good shot values; and if it doesn't it won't.

But some of it must be a lack of talent/insight/experience on my part, i.e. I don't know how to recognize and remember and evaluate a golf hole/course via the various rating criteria. I can't do it very well even in absolute terms (a single course), let alone in relative terms (comparatively, one course 'against' another).

It may also be that this is simply not how my brain works, i.e. not how I experience or think about golf courses. I'm not very good at compartmentalizing (I sometimes wish I was a lot better at it) -- all the aspects and qualities of any golf course I play seem to wash/blend in together for me.

Not until later, when I'm trying to talk about it on here, am I able to think clearly about the constituent parts. I find that, upon first playing a golf hole, I either like, dislike, or am indifferent to it -- right away, and as a whole, without separating out shot values from scenery from challenge etc.         

All of this is not to say that others - like you - can't do it; only that I can't.

And if I were to conclude/speculate, I'd suggest that maybe the issue many have with the GD rankings is not only the 'collective' nature of those rankings; it is also that those rankings reflect the tastes and temperaments and experiences of a particular sub-set of golfer, i.e. the one who can do what I can't.

Is that any different/better/worse than any other collective list or rankings currently available? Aren't all those lists/ranking also reflective of a particular sub-set of golfers, just like the GD one is? I don't know the answer. 

Indeed: maybe even a singular/personal/subjective list and ratings, like TD's Confidential Guide, reflect his ability to see and appreciate aspects of a golf course that I never can or will.   

So, what does/should Tom's ratings really "mean" to me? Or Sean Arble's? Or Golf Digest's?  The only difference I see - and I think it an important one - is that the GD rankings have a patina of 'objectivity' that come both with the collective nature of the overall scores/rankings and with the numeric aspects of the criteria.     

Peter


 

[Tommy Williamsen]

But it's your formula I disagree with, both in concept and in various specifics.  So why would I do any exercise that uses it?  You might as well ask me to go to your church a few times to see if it changes my mind about my religion.  (No offense intended; that was the metaphor I came up with before I realized whom I was addressing!)

I understand your trouble with the formula. I was just hoping that folks might better understand why the rankings might be as they are. By the way, I might try to change one's mind about the quality of a course but never one's religious perspective. I didn't become a pastor to dictate doctrine as odd as that may sound. I try to listen more than speak.

[Mark Bourgeois]

I tried this a few years ago as a New Years Resolution (and posted my new opinions here). Per the spirit of your request, I wanted to make sure I understood how it worked.


The result was the exact opposite of what I think you're hoping to achieve.


The exercise turned me a much sharper, harsher critic of GD. Concluded GD's ranking system is not good for the game of golf. Also, the cognitive dissonance was brutal.


So I really would not recommend this Tommy unless you're hoping to undermine GD!


EDIT
Peter P - the dissonance of applying the formula felt like answering one "do you still beat your wife? Yes or no" question after another.

[Peter Pallotta]

I tried this a few years ago as a New Years Resolution (and posted my new opinions here)...

If you get a chance, Mark, can you post a link to that thread?
I'd be better able to understand it now than I did then.
Thanks
Peter

[Sean_A]

Tommy

While I didn't use GD criteria, I did try rank courses a few years ago and came up blank.  For huge numbers of courses I simply couldn't say with any confidence which courses was better.  The best I could come up with is grouping..deciding which courses fit the best group, 2nd best etc.  Within the group it didn't matter to me where the courses were ranked...I figured there was a sound argument for any which way.  I don't see how using GD criteria would make the process easier or more meaningful. 

I will continue to stick with my "ranking" in terms of travel time/inconvenience to see a course with 1* being very highly recommended.  It seems to make much more sense to me to make suggestions based on how much effort should be expended to play a course. 

3* - don't miss for any reason

2* - plan a significant golf trip around this course

1* - worth an overnight detour

R - worth a significant daytrip...no more travel time than it takes to play and have drinks

r- good fall back on course if in the area/trip filler

NR - not recommended

How I arrive at these recommendations is more by gut instinct, experience and by what people who know more than me think. 

Ciao

[corey miller]

Will someone explain to me the "resistance to scoring" measure?  I guess I am trying to understand how it would not somehow use the course rating and perhaps slope numbers? 

I hate "fair" but then again I hate the argument that something is "unfair"even though i know my definition of "unfair" is totally different than Golf digest. 

[Tommy Williamsen]

Will someone explain to me the "resistance to scoring" measure?  I guess I am trying to understand how it would not somehow use the course rating and perhaps slope numbers? 

I hate "fair" but then again I hate the argument that something is "unfair"even though i know my definition of "unfair" is totally different than Golf digest. 

Cory, GD doesn't define "fair" for us. They figure it's like pornography, we know it when we see it.

[jeffwarne]

 

[Steve Lapper]

But it's your formula I disagree with, both in concept and in various specifics.  So why would I do any exercise that uses it?  You might as well ask me to go to your church a few times to see if it changes my mind about my religion.  (No offense intended; that was the metaphor I came up with before I realized whom I was addressing!)

I wholeheartedly agree with Tom. Your formula is so badly flawed and unable to capture the essence of a course, it's architecture, fun factor, or special interest...so why would I even bother to waste those few moments of my life I will never get back playing a game whose rules and tenets are pathetically limiting?

[Peter Pallotta]

I've done more than my share of complaining about the collective nature of rankings in general and of the GD lists in particular. But now that Tommy asks the question/assigns the task, I find myself confused by the reactions here.

I can't really find fault with the *why* of the criteria; I mean, don't we all want variety and challenge and ambience and interesting shots to play? What I personally can't understand is the *how* of it all; I just don't know how to go about assigning numerical values/scores to distinct components.

Mark says he found it very hard to do; maybe the Seans and Toms and Steves and Jeffs of the world would find it hard to do as well. But I'm guessing that those folks *could* do it if they wanted to. All of which is to wonder out loud:

If only *one* of those guys used the formula and rated/ranked courses accordingly, and if *that* one rater's opinion served as the next GD list, would it be a terrific list that most here would applaud?

[Peter Pallotta]

No one has the time, but I think it would be interesting/instructive for these 10 posters to do the GD rating exercise with their favourite 10 courses (ie essentially, to become the GD panel for a day):
Jim Sullivan
Jay Mickle
Bob Crosby
Ben Cowan (Michigan)
Mark Saltzman
Jeff Warne
Joe Hancock
John Kavanaugh
Terry Lavin
Kyle Harris

[Tommy Williamsen]

But it's your formula I disagree with, both in concept and in various specifics.  So why would I do any exercise that uses it?  You might as well ask me to go to your church a few times to see if it changes my mind about my religion.  (No offense intended; that was the metaphor I came up with before I realized whom I was addressing!)

I wholeheartedly agree with Tom. Your formula is so badly flawed and unable to capture the essence of a course, it's architecture, fun factor, or special interest...so why would I even bother to waste those few moments of my life I will never get back playing a game whose rules and tenets are pathetically limiting?

It is fascinating that a formula is dismissed without even trying it. It is not "my formula." What is "special interest?" Don't you think shot values and memorability get at the architectural merit of a course?
I am not an apologist for the GD formula but think that a few minutes doing the exercise wouldn't hurt anyone who spends hours on this site. I guess I'm a little disappointed that so few would even consider the exercise.

[Tom_Doak]

Tommy:


When I was 21-22 years old and first seeing all these great courses, I played around with various formulas to try and rate them.  (One was to rate each hole from 0-4 points and then assign the last 28 points to other factors.)


The exercise revealed itself as a waste of time.  Instead of making one subjective judgment I was averaging 25 of them, which is way more time consuming but no more "accurate" or defensible. And no matter how I tweaked the formula, courses like Pine Valley and St Andrews were just so far outside convention that it didn't really do them justice.


The GOLF DIGEST formula derives from the introduction to their publisher William H Davis's book on golf courses from the 1970's ( though they didn't actually start putting numbers to it until 1985).  They have stuck doggedly to this formula for 30 years even though Bill Davis would have been the last guy to say he knew the secret to great architecture ... in fact, when he ran the rankings they only had a small committee to make the decisions.  In all those years they have only tweaked the formula by making the criteria more vague, so that you, a longtime panelist, can't explain exactly what Shot Values are, or how Resistance to Scoring incorporates fairness, or how Conditioning balances "firm but receptive" greens.  So why should anyone accept that as gospel?


How many of the 1500 courses I have seen so you want me to go back and break down into the seven categories?  And what do you think I will learn?  I'm not dismissing the formula out of hand; I'm dismissing it because I've seen the results it has produced for more than 30 years, and I thought it was more fair to attribute those results to the formula than to the messengers.

[Sean_A]

I've done more than my share of complaining about the collective nature of rankings in general and of the GD lists in particular. But now that Tommy asks the question/assigns the task, I find myself confused by the reactions here.

I can't really find fault with the *why* of the criteria; I mean, don't we all want variety and challenge and ambience and interesting shots to play? What I personally can't understand is the *how* of it all; I just don't know how to go about assigning numerical values/scores to distinct components.

Mark says he found it very hard to do; maybe the Seans and Toms and Steves and Jeffs of the world would find it hard to do as well. But I'm guessing that those folks *could* do it if they wanted to. All of which is to wonder out loud:

If only *one* of those guys used the formula and rated/ranked courses accordingly, and if *that* one rater's opinion served as the next GD list, would it be a terrific list that most here would applaud?

Pietro

I did try a "formula" ranking and no matter what, when I went by the "numbers" some courses which don't belong get pushed up and some courses which do belong get pushed down. As you say, its not the categories for the most part, its the assigning of numbers.  For the way I look at courses numbers simply don't work because I place a lot of value on originality, variety (which is heavily linked with originality), the walk and how a course makes me feel....most especially do I feel like I want to return.  I also don't look kindly on difficulty for the sake of difficulty....though the magical concept of the championship course which can serve a membership equally well has all but disappeared.  Numbers can't express these ideas in a meaningful way. 

Ciao

[Jim Nugent]

When I was 21-22 years old and first seeing all these great courses, I played around with various formulas to try and rate them.  (One was to rate each hole from 0-4 points and then assign the last 28 points to other factors.)

The exercise revealed itself as a waste of time.  Instead of making one subjective judgment I was averaging 25 of them, which is way more time consuming but no more "accurate" or defensible. And no matter how I tweaked the formula, courses like Pine Valley and St Andrews were just so far outside convention that it didn't really do them justice.

The GD formula/process ranks PV and TOC right near the top of the world, as you see in GD's world top 10 from 2016 (I believe the latest rankings):

1.  RCD
2.  ANGC
3.  PV
4.  CPC
5.  Royal Dornoch
6.  Royal Melbourne West
7.  Shinnie
8.  TOC
9.  Muirfield
10. Merion

That list would draw admiration from most GCAers, I believe.  I think it would (mostly) draw admiration from Tom Doak: he rates 8 of those courses as '10s'. 

My point is that least at the very top, GD's methodology produces a list almost identical to many GCA lists. 

[Tommy Williamsen]

This thread didn't materialize as I hoped it would but it was an interesting exercise nonetheless. The questions that continually seems to emerge are, "Can a course's merits be quantifiably measured?" "Is it possible to rank courses given a set series of criteria?" The folks on this board answer both questions with a resounding, "NO!" I can understand that answer. On the other hand a set series of criteria might at least give us a common way of comparing courses. GOLF WEEK has their own criteria. Would I like to see GD tweak its categories? Probably. I've been on the GD panel for 25 years. When I play a course, any course, I find myself thinking in those categories. It has become natural. I've played about 1500 courses and like everyone have my favorites, some ranked and many not. The categories help me understand the course's merits. Yet they do not help me answer the question, "Do I like this course enough to play it on a regular basis?" That may be the most telling question of all.

[Mark Bourgeois]

Jim N, but for the international rankings doesn't GD abandon its formula? Thought that was the case, looking for confirmation.

Peter P, here's one where I rated Yale and asked for help:
http://www.golfclubatlas.com/forum/index.php/topic,54569.0.html

And here's a post on TOC -- lots of brain pain after writing this:

http://www.golfclubatlas.com/forum/index.php/topic,54539.msg1258823.html#msg1258823

The changes to TOC should improve its ranking because the changes are removing the course's lack of definition. Lack of definition I posit cannot score super-high on shot values because, as Richard Dreyfus might put it, "whose shot values are they anyway"?

Additionally, some of the changes such as eliminating the hollow on 7 were done to improve conditioning.

All in all, I think these changes line up quite nicely with Golf Digest's criteria.

Jim: I would say you have aesthetics too high. For reference, here is what Washington Post columnist Tom Boswell wrote to me about a month ago regarding TOC's aesthetics:

"I look at St. Andrews and see East Potomac Park, but with a whole bunch of ugly bumps and a thousand stupid bunkers that you can't escape. Oh, and it's ugly -- no Potomac River, no Washington Monument framing about 5 or 6 tee shots. Gimme East Potomac every time."

FYI on conditioning East Potomac scores -5.6782. (This is the same course Steve Marino played a few years ago. Shot only -4 and here's what he said: "It's just kind of like you hit it and guess where it goes on this course," Marino said. "I don't think I'd ever shoot over par on a course like this, but I'm not sure I could ever go really low. On nice courses, you know when you hit a good shot that you're going to get rewarded for it. So if you're playing great, you score great. Here, you just never know.")

Oh, last point. TOC isn't even top 3 on Golf Magazine's list: Augusta National has it beat.

[MCirba]

I'm reminded that Joshua Crane tried to come up with a quantitative way to rate golf courses way back when with  some squirrely results.

I'm also reminded of the scene in "Dead Poet's Society" when Robin Williams as the poetry instructor refers his students to the chapter in the textbook where the author comes up with a quantitative method to rate poetry and after a brief overview, asks them each to rip it out and throw it in the trash.

No quantitative measures can tell you how a golf course makes you "feel".   That's the problem, IMHO.

[jeffwarne]

No quantitative measures can tell you how a golf course makes you "feel".   That's the problem, IMHO.

But once GD raters have been told how "to feel"
and thus no one wants to appear stupid by defying the ranings
and
and "outlier" ratings are thrown out, it's tough for things to really change

[Kalen Braley]

I agree with Mike here.


At the end of the day, this is like trying to rank paintings, or poetry, or music, or <insert subjective topic> here.


P.S.  I think the ranking discussions are so polarized is no different than sports radio or espn talk shows...people want to debate things that can never be resolved, because the debate will always rage on regardless of whats said...