Golf Digest's 2013-14 Top 100 Rankings

[Nigel Islam]

If only they measured midi-chlorians, we'd know for certain...

So which architect is the chosen one that will bring balance to the rankings?

[Mac Plumart]

Brad Klein!

He does have a bunker named after him, you know.

[Mac Plumart]

Yep. Hole 18.

[Andy Troeger]

Mark B.,
I believe the reason GD went to 4 decimal places this year is to avoid all the ties that popped up in the 2011-12 issue. I found the ties to be a little silly (and there would have been a tie at #1 this year with two decimal places I believe) so I'm glad they did something to break deadlocks.

David,
In the past (and perhaps this year as well), GD has hired an independent auditing firm to confirm the validity of the results and ensure the things you are suggesting do not occur. Certainly courses certainly have programs to try to elicit positive vibes from panelists, but if they work it would be almost impossible to quantify. They also tend not to stand the test of time. Courses that pop up on the list into a position that is not deserved tend not to stay there for very long.

[Andy Troeger]

Absolutely.  There is no "right" way to do the rankings but there are many "wrong" ways.  Writing a definition of a great course as a model for your rankings, that most people find fault with, is a pretty bad first step.  That's my problem with the GOLF DIGEST ranking.  If they would just admit it's subjective -- as every other ranking does -- no problem.  But they always use the word "objective" in their accompanying article, and attach superiority to their flawed definition of greatness.

Do you really believe that the GOLF DIGEST definition of a great course is correct?

Does anyone?

Tom,
As hard as Ron and the others try to eliminate subjectivity from the process, I think its an impossible goal. Golf courses are closer to art than science in that way. Personally, I don't think categories are necessary to define a great golf course, assuming you have "raters" who know what they are doing. I never go to a course and think "oh this course has great shot values" as I'm playing hole #7. When I sit down later to rate shot values then I think about it, but I tend to look big picture when actually playing the golf course. That said, I do use categories for my personal list, in part because I've always done it that way and don't feel like changing it now that I've got numbers inputted for 400 courses. Its my list and I like the results...so no reason to mess with it.

As far as the method to define greatness, I don't mind the Digest categories, but would not rate them all equally while doubling shot values. The "Andy scale" is as follows:

100 points total
10 points shot values
7 points playability
7 points resistance to scoring
16.5 points design variety
13.5 points memorability
10 points aesthetics
6 points conditioning
6 points ambience
4 points walkability
20 points FUN

The "FUN" obviously is totally subjective and is basically my "how much do I want to go back?" scale. Design variety, memorability, and FUN are very highly correlated for me, but I have to rate everything anyway so I include all the categories in some form. Walking is only four points, but I find walking to be more fun than riding, so it also shows up in other places too.

[Anders Rytter]

Absolutely.  There is no "right" way to do the rankings but there are many "wrong" ways.  Writing a definition of a great course as a model for your rankings, that most people find fault with, is a pretty bad first step.  That's my problem with the GOLF DIGEST ranking.  If they would just admit it's subjective -- as every other ranking does -- no problem.  But they always use the word "objective" in their accompanying article, and attach superiority to their flawed definition of greatness.

Do you really believe that the GOLF DIGEST definition of a great course is correct?

Does anyone?

Tom,
As hard as Ron and the others try to eliminate subjectivity from the process, I think its an impossible goal. Golf courses are closer to art than science in that way. Personally, I don't think categories are necessary to define a great golf course, assuming you have "raters" who know what they are doing. I never go to a course and think "oh this course has great shot values" as I'm playing hole #7. When I sit down later to rate shot values then I think about it, but I tend to look big picture when actually playing the golf course. That said, I do use categories for my personal list, in part because I've always done it that way and don't feel like changing it now that I've got numbers inputted for 400 courses. Its my list and I like the results...so no reason to mess with it.

As far as the method to define greatness, I don't mind the Digest categories, but would not rate them all equally while doubling shot values. The "Andy scale" is as follows:

100 points total
10 points shot values
7 points playability
7 points resistance to scoring
16.5 points design variety
13.5 points memorability
10 points aesthetics
6 points conditioning
6 points ambience
4 points walkability
20 points FUN

The "FUN" obviously is totally subjective and is basically my "how much do I want to go back?" scale. Design variety, memorability, and FUN are very highly correlated for me, but I have to rate everything anyway so I include all the categories in some form. Walking is only four points, but I find walking to be more fun than riding, so it also shows up in other places too.

Why not just have 100% Fun? I mean, maybe i'm stating the obvious but, "How much do i want to get back?" is more or less the definition of what a good golf course should be to me.

[Andy Troeger]

Anders,
Because I'm a bit of a math dork and that would be too easy. And because I have to go through the exercise for Golf Digest anyway. Realistically, that's what it comes down to though.

[David Davis]

David,
In the past (and perhaps this year as well), GD has hired an independent auditing firm to confirm the validity of the results and ensure the things you are suggesting do not occur. Certainly courses certainly have programs to try to elicit positive vibes from panelists, but if they work it would be almost impossible to quantify. They also tend not to stand the test of time. Courses that pop up on the list into a position that is not deserved tend not to stay there for very long.

Andy,

Thanks. If they do great. I was clear to state that I was not accusing GD because I don't really know them at all. Here in The Netherlands we don't get their magazines and if we do I've never seen them.

My question to you then having spent many years working in media and advertising would be, which independent auditing firm was utilized, what did they control exactly, are they an advertiser or could they have paid for their blurb? That's if I wanted to play devils advocate, however I don't. Just saying, I can't tell you how many internet publications (and I know there is a difference between GD and your average internet publication) say they have x # of unique monthly readers and that this is independently audited and guess what, it's not but it curtails questioning and interests advertisers. Anyway please don't go there, I don't mean to pick it apart and again GD might well be far more ethical in what they are producing than other publications.

The greater questions are the ones the other guys like Tom D. have brought up. Resistence to scoring is simple and if that's a big factor I'm playing day in and day out at the best course in the world hands down ;-) well ok, top 10 to be fair.

 

[Mark Bourgeois]

Andy,

They carry it out to thousandths now? Really?

[Anders Rytter]

Anders,
Because I'm a bit of a math dork and that would be too easy. And because I have to go through the exercise for Golf Digest anyway. Realistically, that's what it comes down to though.

I know what you mean regarding the math issue, sometimes i can't help my self either and have to calculate stuff,

But anyway, would be interresting to see the correlation between you 80% and the 20% fun. Ideally they would match...  i guess.

[Andy Troeger]

Anders,
When I took stats I tested some of the concepts on my lists and there was something like 99% correlation between my rankings and the FUN category. Good enough for me 

David,
I couldn't tell you the specifics, however, my feeling is that the editors go to a lot of trouble to make sure these rankings are accurately compiled. I understand what you are saying, but I'm not concerned personally that it happens here.

Mark B.,
We also submit ratings with decimals. I use tenths. If most panelists are doing that, then publishing with decimals doesn't seem out of the ordinary to me. Its kind of the nature of the way the process is run.

[Tom_Doak]

The "Andy scale" is as follows:

100 points total
10 points shot values
7 points playability
7 points resistance to scoring
 

Andy,

I still have never read a definition of "shot values" that I really like, but the juxtaposition you made above made me think: 

Are shot values not just the intersection between playability and resistance to scoring?  Getting both of them right instead of missing to one side?

I think many Digest voters believe that Shot Values are entirely about the difficulty of individual shots, because there is such a strong correlation between the two scores throughout the voting.

[Andy Troeger]

Tom,
I think there is some variation in how panelists interpret "shot values," no matter how many times Ron has explained it to us. The Digest definition is "How well does the course pose risks and rewards and equally test length, accuracy and finesse?"

From the definition, there are two parts that I will put into my own words...
1. Is the course strategic? (risks and rewards)
2. Does the course create interest/challenge through situations that include length, accuracy, thought, and the short game?

Which leads me back to a simplified question...how well does the course present a variety of strategic situations throughout the round?

So for me shot values should stand on its own but have more correlation with design variety than either playability or resistance. I think its the strategy category, not necessarily correlated to difficulty.

[Sean_A]

I will never get the idea of number crunching as a way to decide the best courses.  The idea is so flawed in its conception that inevitably there will be unsatisfactory results.  I think what likely ends up happening for the amateur crunchers out there is they alter formulas to fit their idea of what is good.  In fact, isn't that essentially what top 100 mag editors did for years?  Dubious categories created to even things out along their way of thinking?

Its a list, like any other list. 

Ciao

[Mark Bourgeois]

Andy,

Raters are given the latitude to decide whether to give, say, a 7.7 vs a 7.8? That's a 1 percent difference. What would be some examples of that?

Personally, I would go exactly the opposite way and restrict the scale to fewer intervals. Make the gaps wider to make it easier to draw distinctions but also to be more intellectually honest with my readership ie no false precision. (Studies show humans are more likely to believe in the accuracy of something if a number is attached and furthermore their faith correlates strongly positive with yhe apparent precision of yhe number in question. (Eg they believe a number like 7.8 more than 8.)

[Andy Troeger]

Mark,
I often compare courses to others that I think are similar on certain categories. I don't really want to use specific examples, but I basically use 1-100 by using tenths instead of a 1-10 scale. With only 1-10 as options, given that most of the courses being considered are at least a 5 and almost no course is absolutely perfect in any category (a 10), then I think the decimals are critical to differentiate somehow between the hundreds of courses in that 6-8 range. Giving them all 7's doesn't really solve anything. We aren't required to use the decimals, just allowed.

[Peter Pallotta]

I grew tired of rankings/ratings a long time ago, and yet I've never grown tired of Andy Troeger's posts, and in fact I appreciate them more with each passing year. Strange...

Peter

[Andy Troeger]

Thanks Peter, although I do apologize if you've suffered through all 3,889 of them! Goodness knows I don't have any new material left...

[Tom_Doak]

From the definition, there are two parts that I will put into my own words...
1. Is the course strategic? (risks and rewards)
2. Does the course create interest/challenge through situations that include length, accuracy, thought, and the short game?

Which leads me back to a simplified question...how well does the course present a variety of strategic situations throughout the round?

So for me shot values should stand on its own but have more correlation with design variety than either playability or resistance. I think its the strategy category, not necessarily correlated to difficulty.

I like your definition.  I wish more of the GOLF DIGEST panelists actually thought about it as you are saying.  But that's not what the numbers would indicate.

Let's assume for a moment that everyone did it your way, and let's also assume that DIGEST will print the numbers for each of the top 100 courses in each category when the whole list comes out.

If shot values = strategy, which top-100 courses should be rated highest for strategy?  Then we'll see which are rated highest in the poll for comparison.  I assume you've got your own list already since you have rated each course on each attribute.  I will have to think a while about my own list of "most strategic courses".

[Tommy Williamsen]

From the definition, there are two parts that I will put into my own words...
1. Is the course strategic? (risks and rewards)
2. Does the course create interest/challenge through situations that include length, accuracy, thought, and the short game?

Which leads me back to a simplified question...how well does the course present a variety of strategic situations throughout the round?

So for me shot values should stand on its own but have more correlation with design variety than either playability or resistance. I think its the strategy category, not necessarily correlated to difficulty.

I like your definition.  I wish more of the GOLF DIGEST panelists actually thought about it as you are saying.  But that's not what the numbers would indicate.

Tom,
There may be another way to look at it.  Maybe we do think about it and because we do the numbers come out the way they do.

[Tom_Doak]

Tommy:

Let's test it out.  Go ahead and list your top 10 strategic courses from the GD top 100, and let's see if they score higher in Shot Values than the 10 courses that also scored highest in Resistance to Scoring.

[Tommy Williamsen]

Tommy:

Let's test it out.  Go ahead and list your top 10 strategic courses from the GD top 100, and let's see if they score higher in Shot Values than the 10 courses that also scored highest in Resistance to Scoring.

Ok, I'll bite.  I don't have to look far. Pine Valley, Cypress Point, Merion, Sand Hills, NGLA, Crystal Downs, San Francisco Golf Club, The Ocean Course, Baltusrol Upper, and I'll pick Ballyneal.  I've played 77 of the top 100. Except for the Ocean Course most of the courses are not overly long.  That's probably because at 65 I my tee shots land sooner and sooner.  Regardless, all those courses require well thought out strategy and good execution.

[Jim Colton]

Here are the category scores from last time:

http://www.golfdigest.com/golf-courses/2011-05/100-greatest-golf-courses-methodology

I did this little breakdown last year of how much of a given course's total score was driven by one category versus another. It got pretty interesting when I compared one category versus another. Here was Shot Values vs Resistance to Scoring. The correlation in scores for these two categories was 82%

HIGHEST                             --     LOWEST   
1.   Shoreacres   1.03%   --   Ocean Forest   -0.79%
2.   Cypress Point   1.01%   --   Bethpage Black   -0.77%
3.   Fishers Island   0.74%   --   The Ocean Course   -0.74%
4.   National Golf Links of America   0.69%   --   Spyglass Hill   -0.69%
5.   Maidstone Club   0.69%   --   Butler National   -0.66%
6.   Somerset Hills   0.64%   --   Hazeltine National   -0.62%
7.   Shadow Creek   0.61%   --   Oakland Hills (South)   -0.54%
8.   Monterey Peninsula (Shore)   0.54%   --   Winged Foot (West)   -0.51%
9.   Garden City   0.50%   --   Rich Harvest Links   -0.47%
10.   Ballyneal   0.47%   --   Baltusrol (Lower)   -0.45%

[Kevin_Reilly]

Tommy:

Let's test it out.  Go ahead and list your top 10 strategic courses from the GD top 100, and let's see if they score higher in Shot Values than the 10 courses that also scored highest in Resistance to Scoring.

Ok, I'll bite.  I don't have to look far. Pine Valley, Cypress Point, Merion, Sand Hills, NGLA, Crystal Downs, San Francisco Golf Club, The Ocean Course, Baltusrol Upper, and I'll pick Ballyneal.  I've played 77 of the top 100. Except for the Ocean Course most of the courses are not overly long.  That's probably because at 65 I my tee shots land sooner and sooner.  Regardless, all those courses require well thought out strategy and good execution.

Tommy, it might be interesting (and illustrative of your point of view) to hear your thoughts on the holes or shots at SF Club (just to pick a course from above) that require well thought-out strategy. 

[Tom_Doak]

Here are the category scores from last time:

http://www.golfdigest.com/golf-courses/2011-05/100-greatest-golf-courses-methodology

I did this little breakdown last year of how much of a given course's total score was driven by one category versus another. It got pretty interesting when I compared one category versus another. Here was Shot Values vs Resistance to Scoring. The correlation in scores for these two categories was 82%

HIGHEST                             --     LOWEST   
1.   Shoreacres   1.03%   --   Ocean Forest   -0.79%
2.   Cypress Point   1.01%   --   Bethpage Black   -0.77%
3.   Fishers Island   0.74%   --   The Ocean Course   -0.74%
4.   National Golf Links of America   0.69%   --   Spyglass Hill   -0.69%
5.   Maidstone Club   0.69%   --   Butler National   -0.66%
6.   Somerset Hills   0.64%   --   Hazeltine National   -0.62%
7.   Shadow Creek   0.61%   --   Oakland Hills (South)   -0.54%
8.   Monterey Peninsula (Shore)   0.54%   --   Winged Foot (West)   -0.51%
9.   Garden City   0.50%   --   Rich Harvest Links   -0.47%
10.   Ballyneal   0.47%   --   Baltusrol (Lower)   -0.45%

Jim:

This is great, except that I don't understand the numbers you've posted.

I presume the "highest" scores are for courses that had higher numbers for Shot Values than for Resistance to Scoring?  I guess this only because several of them are short courses that probably have low Resistance to Scoring, so surely there is less correlation for those courses or they never would have got into the list.

I think it's important to examine the raw scores here ... whether, say, Butler National's "Shot Values" [while low in comparison to its Resistance to Scoring] are still higher than the Shot Values for Somerset Hills, where the correlation is the other way around.  Because it's the raw numbers that add up to the total that puts you in or out.