Ratings: Where do the differences emerge?

[Scott Warren]

All this talk about ratings of late got me interested to look at the differences between the lists and ask some of the raters with knowledge of both Golf Digest and Golf Mag systems how they come to differ so greatly in some respects but remain so similar in others.

I chose some courses and compared them. GD ranking is first, then Golf Mag.

Similar/Same
Pine Valley: 2, 1.
Cypress Point: 4, 2.
Shinnecock: 3, 4.
Pebble Beach: 6, 5.
Merion: 7, 7.
Olympic Lake: 26, 25.
Kittansett: 58, 58.
MPCC Shore: 72, 72.

Some difference
NGLA: 15, 9.
Fishers Isl: 9, 20.
The Country Club: 18, 23.
Bethpage: 29, 22.
Riviera: 31, 19.
Plainfield: 71, 61.

Big difference
Friar's Head: NR, 21.
Maidstone: 86, 37.
Valley Club: NR, 51.
Newport: NR, 70.
Yale: NR, 76.
Myopia: NR, 78.

So why are some placed so similarly by the different groups and yet others are so far apart?

[Mac Plumart]

Scott...

I did a fairly robust analysis of Golf Digest vs. Golf Mag rankings awhile ago.  

I found that Digest seemed to favor difficult courses, most likely due to their "shot values" and "resistance to scoring" criteria.  I think that might be why they don't rate Maidstone, but seem to like a harder course like Victoria National.

Also, Raynor courses seemed to be an area of discrepancy as well.

[Tim Martin]

Scott-If you were keeping track of the current "Who is qualified to rate golf courses " thread it would be clear to you that the Golf Digest raters were not comped at Friars Head, Maidstone, Valley Club, Newport , Yale, and Myopia.  

[Scott Warren]

Mac,

Can you link me to that examination you did of the two lists?

[Mac Plumart]

The analysis is actually in a spreadsheet(s) that I keep on my computer.  I did write about it on my website...

http://www.mrpgolf.com/controversial_designers.html

But to be frank, I think the writing stinks and is hard to follow.  Maybe you can get one or two useful nuggets out of it, however.

Sorry, I don't have more to share.

[Tom_Doak]

Scott:

So how come your comparison of the "Big Differences" between the GOLF DIGEST and GOLF Magazine lists only highlights the courses which are rated higher by GOLF Magazine?  What about the crazy courses that are in the DIGEST top 100 that are omitted by GOLF?  [Rich Harvest Farm, et al.]

I am not saying the GOLF Magazine list is perfect -- far from it.  But DIGEST's formula is a prescription for disaster, and it starts to show itself the further you get down their list, not to mention what is implied just off the list in the state-by-state rankings.

[Scott Warren]

Tom,

I just chose those courses at random - places I have visited or have an interest in. They're not selected examples from a larger pool (ie. I had no idea how either magazine rated any of them before I chose the courses). I am sure both have their anomalies.

Just from my own experience with a few of those courses, I seem to agree with Golf Magazine when it comes to those I have played, but moreso I just found it curious that two different pools using different rating criteria had some courses so close to the same mark and - as you would expect with different raters/criteria - many so far apart.

[Jim Colton]

Scott,

  I think Mac's got it all pretty well spelled out on his site, but I think they differences are due to very fundamental differences in the approach.  Golf Digest uses its broad, average Joe panel and uses a very specific formula for deciding what makes a golf course great.  Golf Mag uses a smaller, expert panel and relies on each panelist to determine what makes a golf course great.  Those two approaches couldn't be much more opposite.  I think Golf Magazine's approach is better and leads to a better list.  Or at least it's more in line w/ my viewpoint, which could be only because I value Resistance to Scoring, Conditioning and Ambience less than Ron Whitten and/or a typical GD panelist.

[Andrew Summerell]

Scott,

I’m interested in what seems to be a lack of appreciation for the more traditional in the GD list, especially with the courses you have listed that I have seen, played & love. Fishers Island is #9 in GD, but Maidstone is 49 spots up the list. How does Friar’s Head not make the list?

Scott, how do some of the well ranked modern courses differ between the two lists?

[Jim Nugent]

I think the major U.S. golf magazine lists are more similar than they are different.  On all of them, the top 20 is mostly the same courses, though in somewhat different order.  e.g. 16 of GM's top 20 U.S. courses are also in GD's top 20.  I think our GCA list is also generally similar to their world lists: something like 17 courses in GM's top world 20 are on our GCA top 20.  

There are around 16,000 golf courses in the U.S., and around 26,000 in the world.  Yet a big majority of the same courses populate the best courses lists.  Whoever the raters are, whatever criteria they use to rankk courses, they are mostly drinking the same flavored koolaid.  

 

[Joel_Stewart]

Scott-If you were keeping track of the current "Who is qualified to rate golf courses " thread it would be clear to you that the Golf Digest raters were not comped at Friars Head, Maidstone, Valley Club, Newport , Yale, and Myopia.  

This is completely false.  I'm a GD panelist and have been comped at every one of those courses except for Friars Head and there the member paid my green fees.  Don't make up shit you know nothing about.

[Niall C]

Scott

How different are these ratings in reality ? I would imagine that when you get down to the second fifty top courses their ratings must all be fairly similar such that any small deviation would correspond to a large shift in the rankings ?

Not sure I explained that too well. Anyway I trust the sun is shining in Oz and hope you're at least getting to see the golf course.

Niall

[Tim Martin]

Scott-If you were keeping track of the current "Who is qualified to rate golf courses " thread it would be clear to you that the Golf Digest raters were not comped at Friars Head, Maidstone, Valley Club, Newport , Yale, and Myopia.  

This is completely false.  I'm a GD panelist and have been comped at every one of those courses except for Friars Head and there the member paid my green fees.  Don't make up shit you know nothing about.

Joel-Read the other thread and then take notice of the  emoticon. It was meant as a joke. Please don`t take yourself so seriously. One thing that I do know something about is that you don`t have much of a sense of humor.

[Jim_Kennedy]

Jim Nugent,
There may be 16k courses in the US, but I don't know how the conclusions of the raters can be seen as drinking the same Koolaid, there just aren't that many exceptional places to choose from in the first place.
 
If the number of different entries from all three top 100 lists was totalled it couldn't be more than 300, more likely it is somewhere around 200 (I haven't checked). Even if there were 480 courses in the US worthy enough to include in one of these top 100 rankings it would only represent 3% of what's 'avaliable' to rate.
      

[scott_wood]

factoid: since Golf Digest published their first "top 100" in 1969, and Golf ( I believe, without checking)
in the mid/late 80s, there have been  269 different courses on those two publication's lists........

if you include Golfweek, which is the latecomer to "The Lists" in '97,
and really has 200 Top 100s,
there are 427 different courses.....

whatever your personal take on "Raters/Rankings",   
there sure are a lot of wonderful places to play this game we love.... 

[Jim_Kennedy]

There are many, many wonderful places to play, but if over the past 41 years only 427 courses overall have made it to top 100 lists in the four categories, with 100 of those being courses whose age guarantees a very closed shop in the classic category, that leaves only 327 modern courses out of a pool of ca.15,900 that were exceptional enough to make it to the three remaining lists.

That's not Kolaid sipping. If anything it reveals that greatness is special and that it takes something quite good to make it into that realm, and stay there......

.....and of course there isn't going to much movement at the top. If the top 20 picks from the 4 categories, over say the past 10 years or so, were listed here, what courses would replace them? My answer: zilch, because there aren't any unknowns out there.   

[Mac Plumart]

Scott,

I’m interested in what seems to be a lack of appreciation for the more traditional in the GD list, especially with the courses you have listed that I have seen, played & love. Fishers Island is #9 in GD, but Maidstone is 49 spots up the list. How does Friar’s Head not make the list?Scott, how do some of the well ranked modern courses differ between the two lists?

Andrew, if memory serves me correctly we talked about his a few months back and I think the conclusion was the Friar's Head didn't have enough rater visits to qualify it for National Top 100 list.  It did qualify for the state list however.  I am not familiar with all the nuances of Golf Digests specifics regarding # of visits to qualify, but I am fairly confident the gist of what I am saying is correct.

[Mac Plumart]

If the number of different entries from all three top 100 lists was totalled it couldn't be more than 300

Jim, it is 218...and yes, I am a total geek!!!   

[Mac Plumart]

If anything it reveals that greatness is special and that it takes something quite good to make it into that realm, and stay there......


Is it that these Top 100 courses are so good or is that there are a lot of courses that are really not good?  I've played a number of "great" courses and had many criticisms of them, but when I try to think of what might be an appropriate place to rank them they end up being very high because their competition is so weak. 

I am sure there are many reasons why this is, not the least of which is that not every course is designed to be "great" or Top 100 level.  But I think thay there is A LOT of room for someone with the mindset of designing great courses to make a huge push up the Golfweek Modern 100 (at a minimum).  First off, look at that list and the Golfweek Top 100 modern GB&I.  If someone with deep pockets and a strong desire to build great courses comes along...with enough sense to get people to work for him that actually know how to make GREAT golf courses, they will dominate those rankings.  Hey, what is that guys name, Mike Keiser, yeah, I think that is it...he is doing just this!

Who is next?

[Matt_Ward]

Let me point out that aggregate ratings -- whether carried out by Digest or to a lesser degree by Golf Mag because the rater list is much smaller and likely has more contacts -- start to show weaknesses once you move outside the top 40-50 courses.

When you blend numbers together in the fashion that both carry out -- what emerges as "great" courses worthy of top 100 status is a group think consensus that likely has made such room because of the compromised voting process that takes place. No doubt there are many fine courses but when voters are isolated and you can't really have cross comparisons of a real and direct manner -- you get melded inclusions of courses -- many of which are completely forgotten in a short time and replaced by others who are no less forgotten when other "newer" layouts emerge.

[Jim_Kennedy]

Mac,
Thanks for that number. Mathematically, 300 would have to be the the maximum number for 3 lists (Duh) and off the top of my head I said that 200 was more likely.

I believe the 'chosen ones' are that good, and stand up for so long because of that.
Below are the lists and on them are the top 20 courses from each. The number preceding the name is how many lists each was on. The total number of courses is 42. I'm not going to judge the veracity of the lists, only the idea that the courses stay there because everyone is drinking from the same pitcher of Koolaid. If that concept was true then I'd like to see what's going to replace this lot, and if you did replace this lot with 42 others, could you do it again? There just aren't enough golf courses that rise to top 20 'status',  therefore the "Koolaid" doesn't have any water in it.      

Golf Magazine
3 Pine Valley
3 Cypress
3 Augusta
3 Shinnecock
3 Pebble
3 Oakmont
3 Merion (East)
3 Sand Hills
3 National
3 PacDunes
2 Pinehurst #2
3 Crystal Downs
3 Seminole
3 Winged Foot (West)
3 Chicago
2 Prairie Dunes
3 San Francisco
2 Oakland hills (South)
1 Riviera
3 Fishers Island

Golf Digest1 Oak Hill
2 Wade Hampton
1 TCC(composite)
2 Muirfield
1 Medinah

Golf Week Modern
1 Whistling Straits
1 Bandon Dunes
1 Ballyneal
1 Sebonack
1 The Golf Club
1 Pete Dye GC
1 Friar’s Head
1 Shadow Creek
1 Old Sandwich
1 Kinloch
1 Spyglass
1 Honors
1 Sawgrass
1 Kiawah(Ocean)
1 Kingsley
1 Chambers Bay

Golf Week Classic
1 Garden City
1 Bethpage Black

Edit: I tend to believe that every one of the lists is somewhat like the record book in other sports, except for Golfweek’s Modern. That one is more like the rookies who will one day (probably sooner that later) start replacing some of the old pros on two of the other three, and eventually on the Classic list after we’re long gone.    

[Jim Tang]

Mac -

With regard to Friar's Head, I believe you are correct.  FH has not received enough ballots (not enough panelists have seen the course) to be considered for the Top 100 list.

[Tiger_Bernhardt]

As you know the magazines have totally different criteria. I think within a given 10 to 15 course grouping it is so subjective as to be silly to point fingers or see as significant.

[Jim Colton]

Mac -

With regard to Friar's Head, I believe you are correct.  FH has not received enough ballots (not enough panelists have seen the course) to be considered for the Top 100 list.

Jimbo,

 Are you sure about that?  Hudson National is 12th in the Best in State and 94th overall in Golf Digest.  Piping Rock is 13th and not in the top 100.  Friar's Head is 14th.  I know there's a minimum number of panel visits required to make the top 100, but would a course with less than 45 visits automatically be placed below the top 100 courses in the state rankings because of lack of votes?

[Matt_Ward]

Jim:

I believe the base minimum is different for state-wide polls. How FH is behind Hudson National astounds me.

Then again the Digest listing for the various states has more situations of this type than you can imagine.