Why GD's ranking system is detrimental to the game of golf

[Carl Rogers]

Why not invent some new categories for courses? ............ The "affordable", "short", "unconventional scorecard"..... like Audubon Park in Vol 2 of the Confidential Guide.

[JC Jones]

I don't have any problem with their definition of conditioning, now, except for their inability to get their panelists to use it.  But I'm still not convinced that conditioning should be a factor in the rankings at all, or at least to the point where people are rating it from 1 to 10.  If the course isn't in reasonable shape to play golf, just mark it down on Shot Values ... they are probably doing that anyway.

Which courses, specifically, do you think are over/under rated on conditioning?  What are their conditioning scores and how do you think the panelists have erred in assigning that score?

I don't know of a panelist nor do I see how a panelist would include conditioning into Shot Values as the two categories are evaluating two entirely different things.

I spoke with a few panelists before writing this piece. One of them recalled his experience at Fox Chapel during the week of this year's US Open. As you recall it rained a good 5 inches. He played Fox Chapel on a Friday, they couldn't mow, it was soggy, standing water. He said to me, how could I score it high given their criteria's definition?


That is the problem is rating conditioning, so much of it is out of the course's hands.

Interesting anecdote though not relevant to Tom's assertion that he doesn't think the Panelists are properly applying the criteria nor to my question to him asking for specificity.

[Peter Pallotta]

Joseph Campbell tells of one version of the 'slaying the dragon' myth. On each scale of the dragon you slay is written a 'thou shalt', ie a proscripton/prescription from the outer world that, adhered to, blinds you to your own unique inner light.
Find your top 10, or at most ask a trusted friend for his. But to set your lights by the suppossed wisdom and objectivity of a collective view, of 'the world's' view is to embrace the thou shalts like a child does/is forced to instead of stepping into your mature self as Nature intends.

I'm sorry but I have no clue what in the hell this means. Could you please explain it like I'm a 6-year old, who speaks English? Thank you.

The average golfers who ignore such lists are the grown-ups; we here who discuss and are guided by them are the children.

[Tom_Doak]

Which courses, specifically, do you think are over/under rated on conditioning?  What are their conditioning scores and how do you think the panelists have erred in assigning that score?

JC:  I don't like throwing anybody under the bus.  Let's just say that the last time I looked, the course of mine that had the HIGHEST score for conditioning, is the one that my friends complain most adamantly about as being soft and slow, having drives plug in the fairway, etc.  Some might guess which one this is, but I would prefer if they didn't share.


Looking at their 2015 published numbers for the top 100, the only course to score above an average of 9 for Conditioning was Augusta National ... it was four tenths of a point ahead of Pine Valley at #2 [which is the biggest separation from the pack for any course in any category], and then Oakmont and Shinnecock Hills came close behind that before another fairly big gap, before you get to the courses that almost no one plays [Alotian, Canyata, Double Eagle], which of course score high because there aren't any divots!  I think those numbers are


(a) heavily influenced by the phenomenon John Kirk described on another thread where one high score drags all the others up;


(b)  not reflective of the conditioning of Augusta for most of the months it's open - it was pretty soggy in January when I played it a couple of years ago:  big pitch marks in the greens when I managed to hit one; and


(c)  reflective of how panelists think they ought to vote, rather than the conditions they saw on the day.




Meanwhile, the lowest numbers for conditioning were Maidstone [by a lot], followed by the TPC at Sawgrass, Bandon Trails and Somerset Hills.  Bandon Trails, when I last played it, was as good a playing surface as you could want, and either it or Old Macdonald is the best-conditioned of the four courses at the resort.  [GOLF DIGEST has Pacific Dunes as the best-conditioned -- see reason (a) above -- but they are all on the low end of the scale!]


If their definition of Conditioning is really "How firm, fast and rolling were the fairways, and how firm yet receptive were the greens on the day you played the course," I don't see how any course at Bandon Dunes scores low at all, unless by "yet receptive" they mean to reward soggy greens.  Maybe it's just the number of divots?  Is that what good conditioning is all about?


Also, we have been consulting at Somerset Hills for several years, and if it's in below-average condition for a top 100 course, the standard must be insanely high.

[Jeff_Brauer]

You? Don't like to throw someone under the bus? You have matured since the first CG.............


As to your conditioning points, I get it.  Your description reminds me of either the voting booth, where you lose interest in the minor races and start voting pretty carelessly, or any customer survey or instructor, which you start out being interested in, but other than a few key points you want to make, probably vote all pretty much the same.


I have read that the average American attention span has dropped to ten seconds, and I believe our critical thinking skills are also dropping.  I suppose that alone would argue for a simpler system.

[Greg Tallman]

If the magazine truly wanted a ranking that reached their core reader then conditioning would be about 95% of the evaluation.

[Jeff_Brauer]

Greg,


I agree.


Which leads to the discussion on any ranking, which is how much do you just let human nature be human nature?  What is odd, is the "single rater" theories want to give a lower number of raters more say to avoid the ignorance of the masses (seems that way, anyway), but presumably, anyone in charge would pick someone presumably well indoctrinated to what they think makes greatness, no?  If you want to steer things to a particular result, you have to really contrive something to combat human nature, methinks.


But, its okay to try to direct architecture rankings towards architecture.  I think GD does, and then publishes their reader surveys, where some nicely average courses get 3-5 stars from average golfers as another guide as to where to play.


In one of my earlier posts to Andy, I meant to mention that.  I asked how many really use the Top 100 to make their decisions on where to travel, given most are not accessible anyway?


Lastly, it may be just me, but I see more independent thinking among golfers anyway.  They can search the net to find courses that fit their games more than ever before, and they seem to do so.


Senior men look for courses under 6000 (from their tees) and don't care so much about 7K, difficulty, etc.
Women are starting to look for shorter courses (sub 4500) and ones that make Women Friendly lists.


Etc.


And, we can't forget that the biggest distance issue in golf is the distance from front door to first tee.......

[JC Jones]

Which courses, specifically, do you think are over/under rated on conditioning?  What are their conditioning scores and how do you think the panelists have erred in assigning that score?

JC:  I don't like throwing anybody under the bus.  Let's just say that the last time I looked, the course of mine that had the HIGHEST score for conditioning, is the one that my friends complain most adamantly about as being soft and slow, having drives plug in the fairway, etc.  Some might guess which one this is, but I would prefer if they didn't share.


Looking at their 2015 published numbers for the top 100, the only course to score above an average of 9 for Conditioning was Augusta National ... it was four tenths of a point ahead of Pine Valley at #2 [which is the biggest separation from the pack for any course in any category], and then Oakmont and Shinnecock Hills came close behind that before another fairly big gap, before you get to the courses that almost no one plays [Alotian, Canyata, Double Eagle], which of course score high because there aren't any divots!  I think those numbers are


(a) heavily influenced by the phenomenon John Kirk described on another thread where one high score drags all the others up;


(b)  not reflective of the conditioning of Augusta for most of the months it's open - it was pretty soggy in January when I played it a couple of years ago:  big pitch marks in the greens when I managed to hit one; and


(c)  reflective of how panelists think they ought to vote, rather than the conditions they saw on the day.




Meanwhile, the lowest numbers for conditioning were Maidstone [by a lot], followed by the TPC at Sawgrass, Bandon Trails and Somerset Hills.  Bandon Trails, when I last played it, was as good a playing surface as you could want, and either it or Old Macdonald is the best-conditioned of the four courses at the resort.  [GOLF DIGEST has Pacific Dunes as the best-conditioned -- see reason (a) above -- but they are all on the low end of the scale!]


If their definition of Conditioning is really "How firm, fast and rolling were the fairways, and how firm yet receptive were the greens on the day you played the course," I don't see how any course at Bandon Dunes scores low at all, unless by "yet receptive" they mean to reward soggy greens.  Maybe it's just the number of divots?  Is that what good conditioning is all about?


Also, we have been consulting at Somerset Hills for several years, and if it's in below-average condition for a top 100 course, the standard must be insanely high.

The problem is that you're expecting the change in definition to have an impact sooner than it realistically could.  I can't remember when fast & firm became an emphasis for Golf Digest but it wasnt a whole lot before the 2015 ballots were collected.  So, the 2015 rankings aren't a great place to start.  Furthermore, there would need to be a wave of ballots after the change to move a course's ranking significantly in just one cycle (i.e. 2015-2017).  Particularly as Golf Digest throws out outliers (which makes John Kirk's theory a difficult one to accept) to prevent huge swings.  So, any changes to rankings happen incrementally and require several ballots cast in the same direction to move the score.  And, that is the overall score, not just a single category.  So, any change to a ranking based on conditioning will have to be significant and, more importantly, wont be noticeable until a sufficient lag time has passed. 

I used to push against Golf Digest's policy of throwing out any outliers but your anecdote about Augusta National seems to validate their reasoning.  After a few years now of living in the south, southern courses tend to play their firmest during the winter months.

[Tom_Doak]

The problem is that you're expecting the change in definition to have an impact sooner than it realistically could.  I can't remember when fast & firm became an emphasis for Golf Digest but it wasnt a whole lot before the 2015 ballots were collected.  So, the 2015 rankings aren't a great place to start.  Furthermore, there would need to be a wave of ballots after the change to move a course's ranking significantly in just one cycle (i.e. 2015-2017).  Particularly as Golf Digest throws out outliers (which makes John Kirk's theory a difficult one to accept) to prevent huge swings.  So, any changes to rankings happen incrementally and require several ballots cast in the same direction to move the score.  And, that is the overall score, not just a single category.  So, any change to a ranking based on conditioning will have to be significant and, more importantly, wont be noticeable until a sufficient lag time has passed. 

I used to push against Golf Digest's policy of throwing out any outliers but your anecdote about Augusta National seems to validate their reasoning.  After a few years now of living in the south, southern courses tend to play their firmest during the winter months.

JC:  Augusta is not so firm if they've had much rain in the winter!  It's not on sand like Hilton Head or Florida.


My understanding was that GOLF DIGEST uses the Conditioning scores ONLY from panelists who have played the last two years; if that's true, why would it take a lag time to have an impact?

[JC Jones]

Tom,

Regarding Augusta.  Since when does water not run down hill?  Moves faster on clay than it does on sand 

If your understanding regarding the conditioning scores is true, then a place such as Augusta would be receiving its conditioning score based on significantly fewer ballots than the 45 necessary for consideration.  But still, even if that were so, it is still only 1 of many categories and thus a .1-4 difference in conditioning over another course would have minimal impact on its final score versus that same course.

Regardless, if your understanding regarding the conditions scores is true then your reference to 2015 rankings and scores is entirely irrelevant as they would not include the conditioning score from the last 2 years.

Nonetheless, there are several courses reputed to be poorly conditioned that continue to climb in the rankings so that category cant be that much of an impact. 

[Steve Lapper]

 Putting the discussion of conditioning aside for a minute, I think this thread's title question begets another, almost larger area of debate....Why GD's ratings are doing long-term macro damage to the state of the game?


  Although they are certainly not the only negative influence, nor a deliberate one, I think it's fair to say that the skew of GD's ratings to resistance of scoring, the uniformity of using very low handicappers and it's premium on conditioning have all combine to encourage a significant number of course developers and operators to build and maintain longer courses, favoring designs that move more earth, and spend more money and use more water to achieve aesthetically desirable conditioning.


  All of these activities combine, joint and severally to drive up the costs to the golfer, private club member, resort of other public player as well as increase the difficulty of an already difficult game......the net effect of which is to further depress both the growth and general health of the game. Of course, GD (just like the USGA...cheap shot I know, but quite fair) has no overt obligation to promote the health of the game, however it is in their best financial interests to not contribute to it's deterioration.


  No question that Ron Whitten, Steve Hennessey, Peter Finch and even Jerry Tarde have never purposefully conspired to enable these negative effects, yet it is, IMO, an undeniable unintended consequence. I only wish that larger voices of the golf industry would recognize this and speak out about it.

[Jim Franklin]

I played with a newer panelist at San Francisco GC last year and he was telling me how awful it was. The conditioning was "nowhere near as good as Kinloch". I told him to read the definition again and then tell me SFGC was awful. He said he didn't care, it was his opinion. I had to tell him that he was rating for Golf Digest not ____________ Digest. It does amaze me that many panelists do not rate according to the Digest criteria but to their own.

In playing The Loop this summer, I know other panelists would have hammered the fact that the turf was not fully grown in. I rated it high because it was firm and fast (and fun). So many people cannot get it into their heads that brown is not bad and that falls back to Augusta.

[Mark_Fine]

Jim,
I said this in another post, it is the not the GD criteria that is flawed, the flaws are in the experience or lack there of in the panelists. 

The quality of ANY ranking is a direct reflection of the quality of the people who are doing the ranking. 

Another example, if you toured London on your own vs touring it with a great guide who knew all the history and nuances of the city, might your opinion change?   Reviewing a golf course is no different.  Some know what they should be looking for and some don't. 
 

[Tom_Doak]

Although they are certainly not the only negative influence, nor a deliberate one, I think it's fair to say that the skew of GD's ratings to resistance of scoring, the uniformity of using very low handicappers and it's premium on conditioning have all combine to encourage a significant number of course developers and operators to build and maintain longer courses, favoring designs that move more earth, and spend more money and use more water to achieve aesthetically desirable conditioning.

  All of these activities combine, joint and severally to drive up the costs to the golfer, private club member, resort of other public player as well as increase the difficulty of an already difficult game......the net effect of which is to further depress both the growth and general health of the game.


...


 I only wish that larger voices of the golf industry would recognize this and speak out about it.

Well it's a little late now, isn't it?  There were a few THOUSAND golf courses built to try and please the GOLF DIGEST definition, which led to the consequences you've described, and now there will be like 10 per year for the foreseeable future.


Also, which larger voices are you looking to speak out?  Most of them cashed in on the way things were, so it would be pretty crappy of them to reverse course now and point their fingers at GOLF DIGEST.  The whole problem with the golf business [or with business in general] is that it's the interest of those inside to drive UP the price, not to drive it down.  Driving it down is long-term thinking of the type most businesses say they can no longer afford.

[Mark McKeever]

  Although they are certainly not the only negative influence, nor a deliberate one, I think it's fair to say that the skew of GD's ratings to resistance of scoring, the uniformity of using very low handicappers and it's premium on conditioning have all combine to encourage a significant number of course developers and operators to build and maintain longer courses, favoring designs that move more earth, and spend more money and use more water to achieve aesthetically desirable conditioning.


  All of these activities combine, joint and severally to drive up the costs to the golfer, private club member, resort of other public player as well as increase the difficulty of an already difficult game......the net effect of which is to further depress both the growth and general health of the game. Of course, GD (just like the USGA...cheap shot I know, but quite fair) has no overt obligation to promote the health of the game, however it is in their best financial interests to not contribute to it's deterioration.

Well put Steve

[Joel_Stewart]

Golf digest says it wants firm and fast and gca still doesn't like its definition of conditioning because it might have rained ?

How they define it and how they vote still look like two different things, to me.  I've looked at the conditioning scores for my own courses on occasion, and their panelists' evaluation of conditioning is way different than mine.  Unfortunately, there aren't any conditioning numbers printed in the magazine to prove or disprove my point.

Thats because Augusta is the benchmark and most panelists don't know how to evaluate Bermuda, Bent and Fescue as an example.  I played Cal Club recently and the condition of the grass was pristine yet fescue fairways will never receive the type of votes that bent fairways have.

[Joel_Stewart]

Mike Clayton discusses that GD panelists vote for best experience.  I've been arguing this for years yet GD doesn't want to change. 


I've always wondered why Whitten doesn't want to change a 50 year old system that has run its course?


In the end this is about money in so many ways.

[Ted Sturges]

Which courses, specifically, do you think are over/under rated on conditioning?  What are their conditioning scores and how do you think the panelists have erred in assigning that score?

JC:  I don't like throwing anybody under the bus.  Let's just say that the last time I looked, the course of mine that had the HIGHEST score for conditioning, is the one that my friends complain most adamantly about as being soft and slow, having drives plug in the fairway, etc.  Some might guess which one this is, but I would prefer if they didn't share.


Looking at their 2015 published numbers for the top 100, the only course to score above an average of 9 for Conditioning was Augusta National ... it was four tenths of a point ahead of Pine Valley at #2 [which is the biggest separation from the pack for any course in any category], and then Oakmont and Shinnecock Hills came close behind that before another fairly big gap, before you get to the courses that almost no one plays [Alotian, Canyata, Double Eagle], which of course score high because there aren't any divots!  I think those numbers are


(a) heavily influenced by the phenomenon John Kirk described on another thread where one high score drags all the others up;


(b)  not reflective of the conditioning of Augusta for most of the months it's open - it was pretty soggy in January when I played it a couple of years ago:  big pitch marks in the greens when I managed to hit one; and


(c)  reflective of how panelists think they ought to vote, rather than the conditions they saw on the day.




Meanwhile, the lowest numbers for conditioning were Maidstone [by a lot], followed by the TPC at Sawgrass, Bandon Trails and Somerset Hills.  Bandon Trails, when I last played it, was as good a playing surface as you could want, and either it or Old Macdonald is the best-conditioned of the four courses at the resort.  [GOLF DIGEST has Pacific Dunes as the best-conditioned -- see reason (a) above -- but they are all on the low end of the scale!]


If their definition of Conditioning is really "How firm, fast and rolling were the fairways, and how firm yet receptive were the greens on the day you played the course," I don't see how any course at Bandon Dunes scores low at all, unless by "yet receptive" they mean to reward soggy greens.  Maybe it's just the number of divots?  Is that what good conditioning is all about?


Also, we have been consulting at Somerset Hills for several years, and if it's in below-average condition for a top 100 course, the standard must be insanely high.

 
Tom,   To me, this speaks more to the competence of their panelists rather than the flaw(s) in their process.  The post above about rating Fox Chapel down on conditioning because of casual water the day the rater was there is another illustration.  At some point, they've got to address who they are asking for feedback...in addition to the flaws in their process.  I don't even subscribe to GD any longer and have always considered their list the worst, but I do understand why it remains important to operators and architects.  Thank God you, Ran et al have updated/expanded the Confidential Guide for folks who might want more thoughtful advice on choosing where to play when travelling.
 
TS

[Ted Sturges]

Golf digest says it wants firm and fast and gca still doesn't like its definition of conditioning because it might have rained ?

How they define it and how they vote still look like two different things, to me.  I've looked at the conditioning scores for my own courses on occasion, and their panelists' evaluation of conditioning is way different than mine.  Unfortunately, there aren't any conditioning numbers printed in the magazine to prove or disprove my point.

 


Thats because Augusta is the benchmark and most panelists don't know how to evaluate Bermuda, Bent and Fescue as an example.  I played Cal Club recently and the condition of the grass was pristine yet fescue fairways will never receive the type of votes that bent fairways have.

 
Why?

[Jerry Kluger]

I am trying to stay out of the conversation because I think it is so petty.  Yes, everyone thinks their opinion is better than some panel and that's ok with me but who says that everyone has to agree that your opinion is the only one that counts.  At least it is a panel and not just one restaurant, theater or art reviewer.  Heck, you can make up your own list but I don't know of anyone other than Tom Doak who put up the money to publish his opinion and at that time how many people really cared about his opinion.  Now he certainly has the credibility because he has demonstrated that he can put his money where is mouth is with respect to course design.  I am going to make a wild guess that the new Confidential Guide is selling better than the original. We are going to blame Golf Digest because of all the difficult courses that were built in the last 50 years - tell that to Pete Dye. Come on, stop being so full of yourself and if you don't like some list or opinion then don't read it or ignore it but don't get all bent out of shape over it.

[Mark Bourgeois]

I am trying to stay out of the conversation because I think it is so petty.  Yes, everyone thinks their opinion is better than some panel and that's ok with me but who says that everyone has to agree that your opinion is the only one that counts.  At least it is a panel and not just one restaurant, theater or art reviewer.  Heck, you can make up your own list but I don't know of anyone other than Tom Doak who put up the money to publish his opinion and at that time how many people really cared about his opinion.  Now he certainly has the credibility because he has demonstrated that he can put his money where is mouth is with respect to course design.  I am going to make a wild guess that the new Confidential Guide is selling better than the original. We are going to blame Golf Digest because of all the difficult courses that were built in the last 50 years - tell that to Pete Dye. Come on, stop being so full of yourself and if you don't like some list or opinion then don't read it or ignore it but don't get all bent out of shape over it.

Isn't your beef with Golf Digest then? Jerry Tarde writes that the rankings have "an enormous impact on architecture, which influences site selection for championships."

[Steve Lapper]

Although they are certainly not the only negative influence, nor a deliberate one, I think it's fair to say that the skew of GD's ratings to resistance of scoring, the uniformity of using very low handicappers and it's premium on conditioning have all combine to encourage a significant number of course developers and operators to build and maintain longer courses, favoring designs that move more earth, and spend more money and use more water to achieve aesthetically desirable conditioning.

  All of these activities combine, joint and severally to drive up the costs to the golfer, private club member, resort of other public player as well as increase the difficulty of an already difficult game......the net effect of which is to further depress both the growth and general health of the game.


...


 I only wish that larger voices of the golf industry would recognize this and speak out about it.

Well it's a little late now, isn't it?  There were a few THOUSAND golf courses built to try and please the GOLF DIGEST definition, which led to the consequences you've described, and now there will be like 10 per year for the foreseeable future.


Also, which larger voices are you looking to speak out?  Most of them cashed in on the way things were, so it would be pretty crappy of them to reverse course now and point their fingers at GOLF DIGEST.  The whole problem with the golf business [or with business in general] is that it's the interest of those inside to drive UP the price, not to drive it down.  Driving it down is long-term thinking of the type most businesses say they can no longer afford.

Tom,


  Nowhere in my post did I suggest that "larger voices" should overtly blame GD or rise up to "drive down" pricing and costs. I do blame GD for trying to sell everyone a "bill of goods" still with their ranking system, and ardently defending the status quo of their analytics and processes. The "Golden Ticket" and it's BS statistical defense by Dean Knuth smacks of this IMO.


 Instead, and idealistically no doubt, I'd like to see the USGA, PGA, PGA Tour and manufacturers all speak to things that would endorse lowering the bar for the game to evolve constructively (i.e. 9 & 12 holes course conversions, rolled-back ball, uniform access for young golfers, fun practice centers (think mini Top Golf) and certainly more natural minimalist-style courses). All of this is an idealist's dream, but heck, a dreamer's gotta start somewhere!


  Btw.....with our new President-elect adamant that golf is best described as "aspirational," the chances of my wishes ever seeing fruition are very close to the odds of my shooting under par ever again.

[Jerry Kluger]

Mark: My beef is with guys who get so bent out of shape over ratings.  At least GD gives its criteria that they use in determining their ratings. It certainly isn't a perfect system but neither are any of the others.  The Confidential Guide is certainly respected by this group but I know I don't agree with the ratings of all the courses.

[John_Cullum]

Joseph Campbell tells of one version of the 'slaying the dragon' myth. On each scale of the dragon you slay is written a 'thou shalt', ie a proscripton/prescription from the outer world that, adhered to, blinds you to your own unique inner light.
Find your top 10, or at most ask a trusted friend for his. But to set your lights by the suppossed wisdom and objectivity of a collective view, of 'the world's' view is to embrace the thou shalts like a child does/is forced to instead of stepping into your mature self as Nature intends.

I'm sorry but I have no clue what in the hell this means. Could you please explain it like I'm a 6-year old, who speaks English? Thank you.

Nice.

[JJShanley]

[/size]Mark: My beef is with guys who get so bent out of shape over ratings.  At least GD gives its criteria that they use in determining their ratings. It certainly isn't a perfect system but neither are any of the others.  The Confidential Guide is certainly respected by this group but I know I don't agree with the ratings of all the courses.

You may not agree with the allocations in CGtGC, but the format allows for a conversation by the author of the piece setting out why he rates a particular course at a certain level.  While it includes a listing of courses arranged by scale, each volume (if I understand correctly) helps you to make a decision about courses that you may have a chance to play.  It typically makes a statement that the author believes about the course.  You can consider that having played the course, or before and after you play.  I don't think GD allows that form of conversation.