[Kalen Braley]

I think people are expecting too much from what is inherently a very subjective process.

If you want more objective output, you need to have more objective input. The GD process is anything but objective.

Richard,

I would agree in concept to that statement....but I just don't see how this can be accomplished.  How does one size up two courses in an objective manner and have it be worth anything?

A big part of me would rather have someone who gives more weight to completley unmeasurable things like aesthics, playability, views, artistry over one who tries to measure hole length, number of par 3s, and clubhouse sqaure footage.  I guess the trick is finding someone with similar golf tastes/likes/dislikes and going with thier opinion.  Isn't that why we're all here.   

[Anthony Gray]

What are the categories for rating a course for GD?

  Garland I know you have a bias for Golf Magazine since they made you the cover boy.

 

[Garland Bayley]

What are the categories for rating a course for GD?

  Garland I know you have a bias for Golf Magazine since they made you the cover boy.

 

Dude! That's Kalen. Notice the righty swing.

[Kalen Braley]

Right swing, wrong face...i'm much more dashing...

[Richard Choi]

Richard,

I would agree in concept to that statement....but I just don't see how this can be accomplished.  How does one size up two courses in an objective manner and have it be worth anything?

It would be impossible to completely eliminate subjective nature of a poll like this, but you can change the voting process to make it more objective.

A couple of very simple ways to make the voting more objective would include following:

1. Rate every hole and the overall course rating would be a composite of those ratings.

2. Ask more objective questions instead of subjective questions. For example, instead of having a single "Conditioning" category, break it down to tee/fairway/green/rough/surroundings. Ask how firm it is or at least ask whether or not the conditioning is consistent with the prevaling strategies of the hole.

3. Never ask a general question like "ambiance" because that just begs for people to rate courses like ANGC very high.

I believe those kind of changes would go a long way.

However, just because it is more objective does not necessarily mean that you will get better results. For example, the famous US News & World report started out with a very objective ways to rate each colleges and universities. However, when the results came in and schools were ranked, no many of the top Ivy League schools were rated at the top. So they adjusted weights of individual categories until they got the rankings that they were looking for in the first place.

That is not going to be very objective either.

To be objective, you really need to start out before compiling the list on what is and is not important and find a way to make objective ways to figure out how to obtain the important data and not mess with it once you compile the data.

[Rob_Waldron]

Anthony Nysse: I beg to differ with you regarding access to Colonial. I know three of your members who are also Panelists. They serve as hosts (I played as a guest of one of them)to numerous Panelists. Of course the snub rating does surprise me. 

[Jim Colton]

Can Andy or some other GD rater explain the rater process to me?  Do you simply submit a score for each of the GD categories after you play a course, or do you maintain a rating/ranking of courses you've played and submit them en masse?  Do you submit an overall rank/score or does GD simply compute an overall score based on the categories it deems important and the weights that it deems are appropriate?  Are you able to access a list of your previous submitted scores for possible revision upon further reflection or additional rounds?

Since the end objective is to rate courses against each other (Course #40 is better than Course #41), I'd love to see them convert the scores/ranks into votes for or against other courses that the same panelists are rating higher or lower.  Then you can convert the votes into a best estimate that, based on the collective information from all the panelists, Course A is more likely better than Course B.   Arcadia Bluffs may score well because some golfers think it's the best course they've ever seen, which may be true if you've only played public courses in the midwest.  But if Arcadia is still comparing favorably by raters that have also played Kingsley, for example, then that adds more credibility that Arcadia should be rated higher than Kingsley.

[Tom Huckaby]

Jim - when the lists get posted to the GD website, all explanations of the methodology will be on there. GD is very open about the process.

But in the meantime I shall answer the questions I know....

Do you simply submit a score for each of the GD categories after you play a course, or do you maintain a rating/ranking of courses you've played and submit them en masse?

One can do it either way, but from all I've talked to, most tend to do one at a time. But I could be wrong.  In any case there are no rules about this.


Do you submit an overall rank/score or does GD simply compute an overall score based on the categories it deems important and the weights that it deems are appropriate? 

We evaluate very specific criteria with very specific definitions for each course.  We give no overall score.  How the magazine treats these submissions is beyond me but I always thought they just averaged them all... however there is some weighting involved.  I forget exactly how it goes.  This will be published soon enough.  But maybe Andy T. knows.

Are you able to access a list of your previous submitted scores for possible revision upon further reflection or additional rounds?

Yes.

[Bill Shamleffer]

I am not going to give any opinions about this or other specific golf course ranking, but I do want to make two statements that I am surprised I have not yet seen fully articulated on this thread.

Years ago when someone came out with some "significant" shake up to their list of best restaurants in the world, I read a comment that seems quite obvious but gets missed.  These "rankings" typically come from someone who sells magazines or books.  If the list does not change, there is not reason for the public to buy their latest publications.  Those performing these rankings have a strong incentive to shake up their lists and to generate lots of talk about their lists.

So Golf Digest likely says thank you to GCA for this tread.

Although I would counter that if the list creates too much negative discussion there does create the risk of the source of that list losing credibility.  But based on the history of golf magazines and the current state of golf magazines, this risk may be miniscule.

My second comment is that I like these lists more than I dislike these lists.

I dislike seeing courses ruin their good features just hoping for a good ranking, and I dislike courses designed around how to get a good ranking.

Nevertheless, it is interesting to see if there are some courses I was previously unaware of, and try to find out more about them to see why so many others think highly of them.  It is also interesting to think about if some highly considered courses are overrated.

Of course in the end, I hope that I make my own final opinion about what I like and dislike on courses and which courses I like or dislike, despite how those course are ranked.

I have been on the grounds of Augusta National, but I have never played the course.  As a big golf (and sports fan), Augusta must be visited due to the history of the Masters, and if I ever played Augusta National, it would be one of the great golf thrills of my life.  But if I was offered a year of golf at any course, I honestly would choose 16 other courses on the current list before I would choose Augusta.  This is not evidence the list is faulty, it is just my own opinion on what I enjoy from a golf course.

[Bill_McBride]

Any list with some of those courses on it and The Valley Club of Montecito and Pasatiempo and Ballyneal not on it is just so much marketing BS.  IMHO of course! 

[Garland Bayley]

Right swing, wrong face...i'm much more dashing...

We have to get your wife to put back regular mirrors in your bath, not those carnival skinny man mirrors she has above your sink now.

[Anthony_Nysse]

Anthony Nysse: I beg to differ with you regarding access to Colonial. I know three of your members who are also Panelists. They serve as hosts (I played as a guest of one of them)to numerous Panelists. Of course the snub rating does surprise me. 

Rob,
  We discouraged any unaccompanied guests from playing and not having raters out last summer because of the renovation didnt help either...

Tony Nysse
Asst. Supt.
Colonial CC
Ft. Worth, TX

[Mike Vegis @ Kiawah]

Last rankings following course name

1   Augusta National G.C. 3
2   Pine Valley G.C. 1
3   Shinnecock Hills G.C. 2
4   Cypress Point Club 4
5   Oakmont C.C. 5
6   Pebble Beach Golf Links 6
7   Merion G.C. (East) 7
8   Winged Foot G.C. (West) 8
9   Fishers Island Club 16
10   Seminole G.C. 9
11   Oak Hill C.C. (East) 25
12   Chicago G.C. 28
13   Sand Hills G.C. 12
14   Pacific Dunes 14
15   National G. Links of America 13
16   Crystal Downs 10
17   Wade Hampton G.C. 15
18   The Country Club (Clyde/Squirrel) 20
19   Muirfield Village G.C. 18
20   Medinah C.C. (No. 3) 11
21   Prarie Dunes C.C. 30
22   Whistling Straits (Straits) 24
23   Oakland Hills C.C. (South) 17
24   Victoria National G.C. 22
25   The Ocean Course 38 (Yea!!)
26   The Olympic Club (Lake) 23
27   The Club at Black Rock XX
28   Castle Pines G.C. 36
29   Bethpage State Park (Black) 26
30   Baltusrol G.C. (Lower) 44
31   Riviera C.C. 61
32   Pinehurst Resort & C.C. (No. 2) 19
33   Bandon Dunes 31
34   Southern Hills C.C. 32
35   The Golf Club 40
36   San Francisco G.C. 37
37   Butler National G.C. 21
38   The Honors Course 35
39   Sebonack G.C. XX
40   Inverness Club 41
41   Shadow Creek  27
42   Canyata G.C. XX
43   Olympia Fields C.C. (North) 42
44   The Quarry at La Quinta 47
45   TPC Players Sawgrass (Players Stadium Course) 79
46   Rich Harvest Links 51
47   Los Angeles C.C. (North) 34
48   Kinloch G.C. 29
49   Arcadia Bluffs G.C. 46
50   Shoal Creek 50
51   Spyglass Hill Golf Course 52
52   Milwaukee C.C. 48
53   Garden City G.C. 56
54   Cherry Hills C.C. 53
55   Forest Highlands G.C. (Canyon) 45
56   Double Eagle Club 70
57   Pete Dye G.C. 99
58   Kittansett Club 54
59   Dallas National G.C. 59
60   Peachtree G.C. 87
61   Scioto C.C. 58
62   Eugene C.C. 64
63   Flint Hills National G.C. 49
64   Interlachen C.C. 62
65   Winged Foot G.C. (East) 43
66   Congressional C.C. (Blue) 86
67   The Prince Course 39
68   Stone Canyon Club XX
69   East Lake G.C. 84
70   Galloway National G.C. 74
71   Plainfield C.C. 85
72   Montery Peninsula C.C. (Shore) 77
73   Long Cove Club78
74   Shoreacres 88
75   Harbour Town Golf Links 93
76   Aronimink G.C. XX
77   Laurel Valley G.C. 82
78   The Preserve G.C. 95
79   Mayacama G.C. 66
80   Bandon Trails XX
81   Quaker Ridge G.C. 33
82   The Estancia Club 81
83   Eagle Point G.C. XX
84   Sahalee C.C. (South/North) 76
85   Kiawah Island Club (Cassique) 98
86   Maidstone Club 72
87   Sage Valley G.C. 91
88   Somerset Hills C.C. 57
89   Black Diamond (Quarry) 67
90   Tullymore G.C. 83
91   Hazeltine National G.C. 89
92   Calusa Pines G.C. 71
93   Sycamore Hills G.C. 92
94   Hudson National G.C. 90
95   Valhalla G.C. 68
96   Sanctuary 73
97   Ocean Forest G.C. 75
98   Grandfather Golf & C.C. 65
99   Blackwolf Run (River) 69
100   Crooked Stick G.C. XX

[Sean Leary]

Last rankings following course name

1   Augusta National G.C. 3
2   Pine Valley G.C. 1
3   Shinnecock Hills G.C. 2
4   Cypress Point Club 4
5   Oakmont C.C. 5
6   Pebble Beach Golf Links 6
7   Merion G.C. (East) 7
8   Winged Foot G.C. (West) 8
9   Fishers Island Club 16
10   Seminole G.C. 9
11   Oak Hill C.C. (East) 25
12   Chicago G.C. 28
13   Sand Hills G.C. 12
14   Pacific Dunes 14
15   National G. Links of America 13
16   Crystal Downs 10
17   Wade Hampton G.C. 15
18   The Country Club (Clyde/Squirrel) 20
19   Muirfield Village G.C. 18
20   Medinah C.C. (No. 3) 11
21   Prarie Dunes C.C. 30
22   Whistling Straits (Straits) 24
23   Oakland Hills C.C. (South) 17
24   Victoria National G.C. 22
25   The Ocean Course 38 (Yea!!)
26   The Olympic Club (Lake) 23
27   The Club at Black Rock XX
28   Castle Pines G.C. 36
29   Bethpage State Park (Black) 26
30   Baltusrol G.C. (Lower) 44
31   Riviera C.C. 61
32   Pinehurst Resort & C.C. (No. 2) 19
33   Bandon Dunes 31
34   Southern Hills C.C. 32
35   The Golf Club 40
36   San Francisco G.C. 37
37   Butler National G.C. 21
38   The Honors Course 35
39   Sebonack G.C. XX
40   Inverness Club 41
41   Shadow Creek  27
42   Canyata G.C. XX
43   Olympia Fields C.C. (North) 42
44   The Quarry at La Quinta 47
45   TPC Players Sawgrass (Players Stadium Course) 79
46   Rich Harvest Links 51
47   Los Angeles C.C. (North) 34
48   Kinloch G.C. 29
49   Arcadia Bluffs G.C. 46
50   Shoal Creek 50
51   Spyglass Hill Golf Course 52
52   Milwaukee C.C. 48
53   Garden City G.C. 56
54   Cherry Hills C.C. 53
55   Forest Highlands G.C. (Canyon) 45
56   Double Eagle Club 70
57   Pete Dye G.C. 99
58   Kittansett Club 54
59   Dallas National G.C. 59
60   Peachtree G.C. 87
61   Scioto C.C. 58
62   Eugene C.C. 64
63   Flint Hills National G.C. 49
64   Interlachen C.C. 62
65   Winged Foot G.C. (East) 43
66   Congressional C.C. (Blue) 86
67   The Prince Course 39
68   Stone Canyon Club XX
69   East Lake G.C. 84
70   Galloway National G.C. 74
71   Plainfield C.C. 85
72   Montery Peninsula C.C. (Shore) 77
73   Long Cove Club78
74   Shoreacres 88
75   Harbour Town Golf Links 93
76   Aronimink G.C. XX
77   Laurel Valley G.C. 82
78   The Preserve G.C. 95
79   Mayacama G.C. 66
80   Bandon Trails XX
81   Quaker Ridge G.C. 33
82   The Estancia Club 81
83   Eagle Point G.C. XX
84   Sahalee C.C. (South/North) 76
85   Kiawah Island Club (Cassique) 98
86   Maidstone Club 72
87   Sage Valley G.C. 91
88   Somerset Hills C.C. 57
89   Black Diamond (Quarry) 67
90   Tullymore G.C. 83
91   Hazeltine National G.C. 89
92   Calusa Pines G.C. 71
93   Sycamore Hills G.C. 92
94   Hudson National G.C. 90
95   Valhalla G.C. 68
96   Sanctuary 73
97   Ocean Forest G.C. 75
98   Grandfather Golf & C.C. 65
99   Blackwolf Run (River) 69
100   Crooked Stick G.C. XX

We are shocked  that you are happy with Kiawah's rise...

[Tom Huckaby]

I am impressed you have yet to mention that of Prairie Dunes, Sean.....

 

[Sean Leary]

I am impressed you have yet to mention that of Prairie Dunes, Sean.....

 

And I am impressed that you haven't mentioned Chicago GC's rise was due to your rating being "Knuthed" out.. 

[Sean Leary]

I wonder if you put the average scores up next to each course with what they scored two years ago how different the numbers would be?

Take a Sycamore Hills or Sahalee for instance. Any chance that they got so few new ratings that their place just stays the same?

[Andy Troeger]

Jim - when the lists get posted to the GD website, all explanations of the methodology will be on there. GD is very open about the process.

But in the meantime I shall answer the questions I know....

Do you simply submit a score for each of the GD categories after you play a course, or do you maintain a rating/ranking of courses you've played and submit them en masse?

One can do it either way, but from all I've talked to, most tend to do one at a time. But I could be wrong.  In any case there are no rules about this.


Do you submit an overall rank/score or does GD simply compute an overall score based on the categories it deems important and the weights that it deems are appropriate? 

We evaluate very specific criteria with very specific definitions for each course.  We give no overall score.  How the magazine treats these submissions is beyond me but I always thought they just averaged them all... however there is some weighting involved.  I forget exactly how it goes.  This will be published soon enough.  But maybe Andy T. knows.

Are you able to access a list of your previous submitted scores for possible revision upon further reflection or additional rounds?

Yes.

Jim,
Tom's answers are what I would have said as well. I will add that I do my scores on an excel spreadsheet, let them sit a little while to make sure they are to my liking, then submit. Usually I take trips with a few courses involved, so I submit them all at once but don't necessarily do the actual rating at the same time.

Regarding the criteria--this will be detailed on the website when its posted, but assuming nothing's changed all of the categories are given the same weight except for "shot values" which is doubled. The other categories are resistance to scoring, design variety, condition, aesthetics, and ambiance. I wish playability and walkability were still included, but no such luck.

[jkinney]

I must say that over the years I've found the Golf Magazine rankings improving and the Golf Digest rankings deteriorating. By saying so, I am not descending into ad hominem attacks on any raters explaining their methodology on this thread. I'm merely stating a preference for the courses I see on the Golf Magazine list.

More and more of the courses on the Golf Digest list don't make the cut for me at all. Period. And as Forest Gump muttered - that's all I have to say about that.

[Matt_Ward]

Richard:

With all due respect - anyone with a half a twit of brain knows this is a subjective exercise.

The point is how to place a weight on those who are truly national panelists versus those who are more regional / state oriented. When you assign equal weight to raters who are clearly not equals you have problems happening as they do now.

One of the good things that Digest started years ago was assigning courses to different raters. This prevents the reach out and grab your favorite rater to get a definite score approach. Unfortunately, the courses assigned tended to be ones in your immediate area.

I would think that one way to add a bit of value to any score is if the person has played multiple rounds over a course. What often makes the state ratings out of place is when out-of-area people post numbers from a one-time visit and this skew a place either up or down. On the flip side I would tend to place more weight from those who do travel and play a wider array of courses / states than others. This would pertain only to the selection at the national level. 

[Mike Jansen]

Everytime one of these comes out I get disgusted!  Spyglass Hill at #51 is a slap in the face.  I have played 7 other couses in the Top 50 and Spy surpasses most of those (only Cypress is the exception). 

When the raters think that Bandon Dunes is more "Memorable" and better "Manicured", they're smokin something really bad.

Why is it so overlooked?

[Richard Choi]

Matt, I am not knocking the list because it is subjective. I am pointing out that expecting rhyme and reason out of a subjective list is futile.

I highly doubt that limiting the raters to "truly national panelists" is going to change anything. I am guessing people who can get access to ANGC (and rate it #1) can probably play anywhere in the world (and does).

When you have a subjective list, you are at mercy of people's whims. Either you should accept the limitations of subjective lists and enjoy them for what they are, or you should make the process a lot more subjective.

[PCCraig]

This is from a Golf Digest press release from Ron Whitten;

“Although the architectural changes at Augusta National have received criticism over the past few years, our panel consensus ranked it No. 1,” said Ron Whitten, Golf Digest Senior Editor of Architecture. “We evaluate seven different aspects of each course's design, and while it's hard to pinpoint individual reasons for a course’s ranking, I think it's fair to say that Shot Values and Resistance to Scoring were the likely categories that contributed to Augusta’s elevation.”

[Adam Clayman]

Pat,

Geoff Shaq makes the excellent point about that quote. "Shot values" being a made up term that has no bearing on (or meaning) to anyone outside of the panelists, and, "resistance to scoring" when anybody can build a hard golf course. Doesn't put much value for why it's the new #1.

One of the subtexts I sense is how convenient it is for the editor to insulate themselves from direct ridicule by having so many panelists.

[Matt_Ward]

Gents:

I read RW's comments on ANGC -- how does the magazine square the fact the core qualities that Jones / Mackenzie created have been thrown under the bus by what you see today?

Do the editors of the magazine really just sit back and allow the raters to say whatever the results will be?

Adam, the magazine is not "insulated" from "ridicule." They've earned it by placing this result in their pages. As an FYI -- years back Digest got heat for dropping ANGC from their "first ten" of listed courses -- ditto the rapid leap by Shadow Creek to the top ten. Some saw that as gimmicks -- now when I see what they've done with the elevation back to the top of the heap I have to wonder why the inability of the editors to really do more to reform what is happening.


Richard:

Allow me to state that I am not suggesting that "rhyme and reason" can happen either. We are on the same page.

The simple fact is that when you engage the services of several hundred people who are treated as exact equals you will get the hodge podge list you see.

My point on narrowing those who are elevated to the role of "national panelist" is a plus because you have people who can provide the needed cross section comparisons / contrasts. The sheer bulk of most raters are those that are state / regionally focused folks.

Richard, I choose to ignore Digest's list -- the issue is that the magazine touts it as some sort of LIST of immense respect. The issue is that the final product is devoid of credibility given the courses that are listed and those that are missed. I will stress again -- the final rating product of any pub is an opinion -- there is no 100% failsafe answer.

You don't see to understand what I mentioned. The magazines can certainly have a handle on who is a bonafide national versus regional state rater. You weight the vote accordingly. On the flip side -- when you have state ratings -- you give added weight to those who live in the immediate area because they see such courses much more frequently.

Richard, I am not looking to "make the process a lot more subjective" -- I am simply attempting to make sure that the sources who provide such information are fully up to speed on just what is being rated from the get go. Digest has taken the route to add even more raters in the hope that MORE people will add better information and results. The listings produced recentlt show that such an approach is not working.

[Tags:]

Tags:

[Tags:]

Tags:

Call to undefined function theme_linktree()