[Steve Lang]

   SO SHOULD THE MANY GOLF BLOGGERS' OPINIONS BE COMPILED AND RELIED UPON TO INFORM THE MASSES?

[Jeff_Brauer]

Steve,


I have always thought that would be the next logical step, averaging rankings like the College Football rankings are.


Reading the last few responses, I am not sure a stats guys would agree with the idea that they would rather see just one persons rankings.  Generally, the larger sample size, the more valid the statistical result.


Thus, it follows that almost anyone who thinks they prefer to see just one person's rankings is really just trying to justify seeing a ranking that agrees with theirs.  It would take just as much strength of character to objectively view another type of ranking as it would in any other system, so there is no advantage to either there.


My only points in my long posts above were to further flesh out the Golf Digest version to more detail, based on hundreds of better players views.  Perhaps still not a valid sample size.  For that matter, its an attempt to flesh out Tom Doak's definition of shot values, a shot/hole/course (he thinks whole course, actually) that "makes a golfer think first and then execute, if he or she wants to score well."  He and other intuitive supporters think its better to leave it there and to someone's individual perceptions.  It seems equally natural that a Brad Klein or Ron Whitten, in trying to give guidance to hundreds to carry out an assigned task, try to formalize the criteria much more.


In my case, I enjoy parsing those out in more detail on long flights, late night reading or discussion (including visiting this site).  For me, once you state your goal, i.e., making golfers think, the next logical question is, "is there a good/better/best" way to do that in any given situation?  Obviously, a thought could range from a Homer Simpson like "Oh, look, a puffy cloud" to an over analysis of sand depth differences in the left and right green side bunker, which none of us architects could even envision in most cases.


Over 18 holes, and 20,000 courses worldwide, there are many unique answers, so it pays not to drill down too far, for fear of standardization.  But, there probably are some nearly universal principles that can probably be applied.


Besides, John Kirk's OP wasn't really about rankings anyway, just a definition of shot values, so why does this discussion always morph to a comparison of different rankings techniques?

[JESII]

Until Paul Rudovsky posted his 7 thoughts I'd never heard of him (sorry Paul...). I looked him up and he's put in the time to really see a ton.


Any if the magazines could help fill in any blanks on telling us why we should listen to his opinion...and then he can have at it.


I'd bet there are dozens, or hundreds of qualified people whose lists and commentary would be really interesting.

[Jeff_Brauer]

Another interesting point, to me......


Reading Tom Doak's and Sean A's response that "it may produce results that clash with your true feelings about the relative merits of two courses, formed without using any formula at all.  So the question is whether any formula is really good enough to stick to when your gut tells you different."


That sort of sounds like "my mind is made up, don't confuse me with facts" mentality, with potential to have a lot more bias than a point system.  Why is that so great?  If your gut is always right, your unemotional response should bear it out under scrutiny, no?


While that may be too harsh among wishes of good tidings in the new year, I guess it doesn't surprise me at all that the point system of GD gets many more modern courses rated higher. Logical, since they have been designed or re-designed with knowledge of the modern players (with a bias to good players, which is traditional) games in mind.  But, when a Klein or Doak first professes their love for older courses (which is fine and well deserved) they are obviously tilting the scale that way by their internal mental state and thought processes. Not to mention, its in their minds when they pick their collaborators.


Or, put another way, doesn't it make more sense to think objectively or even subjectively first to set criteria, then evaluate courses, rather that set a list of courses you like, and then give them a shot value ranking, perhaps forming an entire system around that bias?  I believe all systems are probably subconsciously set up to reflect the initiators values, no?


In a way, that's what happens, with the predictable result that when one entity comes up with their ranking system, and explain the criteria, another responds with their own system, and slightly different criteria.  In the end, that's all you can do - understand your criteria and be as honest as possible - then put it out there for criticism and discussion.


Hopefully, it remains friendly discussion, and fun discussion, just another way to enjoy golf when its cold, rainy or dark outside......I understand I can not write anything that will change a mind on the debate of intuitive vs. scientific analysis of golf courses.

[John Kirk]

I'm slowly working my way through "Thinking, Fast And Slow" by Daniel Kahneman, which is a broad based book about the mind.  The first quarter of the book is full of information about how the mind makes judgements and evaluations, applicable to those of us who spend time evaluating stuff.  Veering off course into gratitude, what a gift it is to enjoy and evaluate golf courses.  Happy New Year everyone.  Hopefully 2017 will bring prosperity and freedom for more living beings.

In the chapter called "Jumping To Conclusions", Kahneman first discusses his experiences grading student tests, and found that he became uniformly biased, based on the student's answer to the first question.  He would then grade the student's subsequent answers higher.  He calls this the "Halo effect", and learned to correct his bias by grading papers one question at a time.  Whether the evaluation of a first hole affects a course's overall ranking could be the basis for another thread.

He then discusses how to combat the Halo effect by increasing the number of evaluators.  He calls it 'decorrelating error'.  The example given is guessing the number of coins in a jar.  He explains that an individual may badly estimate the number of coins, but the average guess for a large number of estimators will be surprisingly accurate.  He stresses that the observers must not share a bias, that their judgements need to be independent of one another.

I believe golf is intrinsically more complex and interesting if the turf is firm, and there are variable winds strong enough to affect play.  I can make a strong argument for that claim.  But most of my low handicap friends here in Portland, Oregon are accustomed to light winds, with soft ,narrow, tree lined fairways, and greens with modest internal contouring.  They tend to see things differently than I do.  I like the Golf Magazine panel, where 100 relative "experts" of golf design evaluate the courses.  I also liked our own GCA poll conducted several years ago.  But the Golf Digest and GolfWeek lists will be the best barometer for many golfers.  In any case, a large number of votes (100 is plenty to eliminate 'error') will get you closer to the "truth".

Every time I read ten more pages of this book, I seem to find something relevant to the art of evaluation.  I think I'll be a better person for reading it.  And I might have to give up this idea that I possess a cold and superior aversion to bias.

[Paul Rudovsky]

Until Paul Rudovsky posted his 7 thoughts I'd never heard of him (sorry Paul...). I looked him up and he's put in the time to really see a ton.


Any if the magazines could help fill in any blanks on telling us why we should listen to his opinion...and then he can have at it.


I'd bet there are dozens, or hundreds of qualified people whose lists and commentary would be really interesting.

Jim--


Thanks...being anonymous ain't so bad.


Truth of the matter is most golfers don't have the time, inclination, or bad enough cases of insomnia or constipation to read the writings of dozens or hundreds of golf nuts like me (and probably you)...and if they do have the time and are like me, they will want to see a summary of all of the dozens or hundreds of opinions they just read.  And that is what the magazines provide...and what most golfers want to see (or have the time to see).


I will say again that the trend in the quality of the ratings in each of the four major USA listings (including www.top100golfcourses.co.uk) is definitely up...and so long as it continues that way, I think things are getting better.  Again, I an NOT being critical of the folks who shepherded the early ratings (including as I recall Tom Doak for GM)...they were pioneers and finding one's way across fresh snowfall with no footprints to follow is very tough.


Now to throw some gasoline on the fire, I believe that the Golf Week system of two separate lists (Classic and Modern) is now outmoded.  It may have been helpful 15-20 years ago, but no longer for the following reasons:


1.  many of the classic courses have now been restored or renovated by modern architects who now have been able to (and for the most part utilized) the heavy machinery of modern times...


2.  one distinguishing feature of the original classics was they were built on great land, while when the early modern courses were built near major cities (in the 60's - '80's), all the great land was gone or too $$$...since Sand Hills and Bandon opened up new lands that are in isolated areas, modern courses have been built on great land once again, and the land factor, which used to distinguish classic from modern is significantly diminished


One factor that made the classics great that has not changed is that laws and regulations governing wetlands has had an incredible impact on golf architecture. I am not trying to open up a political discussion of the appropriateness of these laws and regulations, but it is important to note that courses like NGLA, Cypress Pt, Newport, Yeamans, Seminole, etc etc could never have been built under todays' laws


Thoughts???


Paul

[John Kirk]

Come to think of it, the Halo Effect may be responsible for the consistency of scores in a Golf Digest evaluation.  Courses tend to have similar scores for all categories of evaluation.

Golf can be divided into two discrete activities, observing and playing.  Observing involves walking the course and watching other shots.  Playing includes the entire pre-shot thought process, execution, and observation of results.  Where my analysis tends to diverge from the common shot values analysis is the visual experience before, during and after the shot.  Also, the amount of time spent playing shots is a small fraction of the time required for a round of golf.

[Tom_Doak]

Come to think of it, the Halo Effect may be responsible for the consistency of scores in a Golf Digest evaluation.  Courses tend to have similar scores for all categories of evaluation.

Yes, but as I said earlier, I think it starts with an overall "gut" opinion of the course rather than the quality of the first hole, or whichever of the criteria GOLF DIGEST presents first on their evaluation form.  However, I freely admit that I am biased against those courses which "start slowly" and too obviously save the best ground for the finishing holes of each nine ...


Someone gave me the Kahneman book for Christmas a couple of years ago, but it gave me a headache and I had to give up on it ... you're a better man than I am!  Happy New Year John!

[Tom_Doak]

My only points in my long posts above were to further flesh out the Golf Digest version to more detail, based on hundreds of better players views.  Perhaps still not a valid sample size.  For that matter, its an attempt to flesh out Tom Doak's definition of shot values, a shot/hole/course (he thinks whole course, actually) that "makes a golfer think first and then execute, if he or she wants to score well."  He and other intuitive supporters think its better to leave it there and to someone's individual perceptions.  It seems equally natural that a Brad Klein or Ron Whitten, in trying to give guidance to hundreds to carry out an assigned task, try to formalize the criteria much more.

One would not have to give guidance to those hundreds if one believed they all had a good understanding of the subject.

[Peter Pallotta]

Fascinating.
Man says: "I found how to get to the Promised Land - come this way" and we say: "Well, that's just your *opinion*, and plus, I'm not gonna follow anyone else's map --  I have to find my own way".
But, man says: "These are the top 20 courses in America, the best golf you can play" and we say: "OMG, I have to find a way to get access to those 20 courses - they're on my bucket list, I gotta play them!"
I know: apples and oranges, but it strikes me as a little quirk of the human psyche. In 'worldly' matters the signposts, roadmaps and external ratifications/confirmations of *others* mean a great deal to us, whether we admit it or not; in matters of real import, on the other hand, it is a point of pride for us that we each find and follow our own road.

[JESII]

Thus, it follows that almost anyone who thinks they prefer to see just one person's rankings is really just trying to justify seeing a ranking that agrees with theirs.  It would take just as much strength of character to objectively view another type of ranking as it would in any other system, so there is no advantage to either there.

Jeff - you've not been paying attention, have you? If I had any motive for preferring an individuals rating list over the collective it's so that I can argue with them, not agree...

[Ulrich Mayring]

Late to the thread, but here's my definition of shot values:

"It's all about the technicalities of the actual golf being played: the course tests a variety of different shots played with different clubs and presents a good mixture of challenges and risk/reward scenarios. The greenkeeping practices and general conditioning must support the type of game designed into the course."

Thus the element of conditioning - sometimes seen outside the scope of shot values - is in my opinion an integral part of it. Conditioning is never an end unto itself and so cannot be a category of its own. I believe it is always to be judged in relation to the types of shots being asked for.

So if you want a tl;dr definition:

"Variety of shots and appropriate conditioning"

Ulrich

[Tom_Doak]

Late to the thread, but here's my definition of shot values:

"It's all about the technicalities of the actual golf being played: the course tests a variety of different shots played with different clubs and presents a good mixture of challenges and risk/reward scenarios. The greenkeeping practices and general conditioning must support the type of game designed into the course."

Thus the element of conditioning - sometimes seen outside the scope of shot values - is in my opinion an integral part of it. Conditioning is never an end unto itself and so cannot be a category of its own. I believe it is always to be judged in relation to the types of shots being asked for.

So if you want a tl;dr definition:

"Variety of shots and appropriate conditioning"

Ulrich:


The main reason I'm against using conditioning as a factor in rating courses is just this.  I build in ground-game options because there are lots of golfers that need them, but I've also built plenty of courses where I come back a few years later and find the conditions softer than I would have liked to see them.


It's one thing to knock a course down a bit because the conditions are soft and sloppy ... I have no problem with that.  But it seems like you're going to knock it down MORE if it was designed well and the conditions are letting it down, as opposed to somebody building a course that can only be played when the conditions are sloppy.  And the latter is just a bad design, so I don't understand why you'd reward it.

[Ulrich Mayring]

Tom,

I see your point and agree, it is unfair to the designer. But I suppose I'm not in the business of rating designs, but golf courses. The Kilmore 9 at Carne last year is a prime example: there was less fairway on the course than the architect intended and that made it less playable. So I am knocking the course down a bit (still a very good rating, though). I did notice that Ally wasn't too happy about the hard rough either.

Another case in point was a Bernhard von Limburger course near Frankfurt (Schloss Braunfels). It was clearly designed for firm and fast and had considerable ground game options built-in. When I played in July they levelled a sur-charge onto the greenfee for "extra water - you won't find a greener course anywhere". I really don't see a way to let this just go as a rater and say "well, Limmy must be spinning in his grave, so let's rate the bones".

Yes, it's possible that a good course destroyed by maintenance practices will be rated lower than a mediocre course that is conditioned to be fun to play.

How do you do it? Suppose you see a great shot, but it's unreasonable to try due to conditioning, do you count it for "shot values"?

Ulrich

[Sean_A]

Late to the thread, but here's my definition of shot values:

"It's all about the technicalities of the actual golf being played: the course tests a variety of different shots played with different clubs and presents a good mixture of challenges and risk/reward scenarios. The greenkeeping practices and general conditioning must support the type of game designed into the course."

Thus the element of conditioning - sometimes seen outside the scope of shot values - is in my opinion an integral part of it. Conditioning is never an end unto itself and so cannot be a category of its own. I believe it is always to be judged in relation to the types of shots being asked for.

So if you want a tl;dr definition:

"Variety of shots and appropriate conditioning"

Ulrich:


The main reason I'm against using conditioning as a factor in rating courses is just this.  I build in ground-game options because there are lots of golfers that need them, but I've also built plenty of courses where I come back a few years later and find the conditions softer than I would have liked to see them.


It's one thing to knock a course down a bit because the conditions are soft and sloppy ... I have no problem with that.  But it seems like you're going to knock it down MORE if it was designed well and the conditions are letting it down, as opposed to somebody building a course that can only be played when the conditions are sloppy.  And the latter is just a bad design, so I don't understand why you'd reward it.

Tom


Even if you find exceptional conditions one way or the other?  I have no problem with slapping a rating down slightly or raising it slightly (sort of decider issues if I am on a borderline) because of unusual conditioning.  As Ulrich says, its the course being rated, not the archie or the design.  That said, its quite rare for me to alter a rating based on conditioning.  It just so happens I did whack Renaissance for its mad rough...sorry 


Ciao

[Tom_Doak]

It's one thing to knock a course down a bit because the conditions are soft and sloppy ... I have no problem with that.  But it seems like you're going to knock it down MORE if it was designed well and the conditions are letting it down, as opposed to somebody building a course that can only be played when the conditions are sloppy.  And the latter is just a bad design, so I don't understand why you'd reward it.

Sean:


I thought my post was pretty clear.  I've never said conditioning should count for nothing, but for me it has to be exceptionally good or bad to make a difference.


One of the problems with rankings and lists, instead of reviews, is that you don't have any room to be clear about why you've come to your conclusion.  In The Confidential Guide, if conditioning has impacted our ranking, we have pretty much always said why ... Portrush is one example in the most recent book. 


But if you knock a course down 25 or 50 places in a ranking because of conditioning, you're making a big assumption that it's always going to be that way, which may or may not be true.  For example, your comment implies that if you'd played The Renaissance Club one year earlier, you would have rated it higher.  That's fine ... but how is someone reading your list going to know which year you played it, or what your objection was?  I have no idea if they're still stuck on the narrow fairways they had for the Renaissance Cup, though I'd bet that sooner or later the members will revolt against them. 

[Ulrich Mayring]

Yep, I absolutely agree on publishing category scores plus writing a textual review plus showing a couple of pictures. If you do that then it doesn't matter whether you create the actual ranking by simply adding up the category scores. Every reader should then have the possibility to make his own informed decision.

I don't believe in rankings where it is kept a secret how they came about.

Ulrich

[Paul Rudovsky]

Tom--


You comments in your last paragraph are IMO an argument for large panels and the law of large numbers.  I see no problem w a after knocking a course down 25-50 places for bad conditioning.  If that is caused by a fluke and the 50 other raters who play all see it in good condition, then it is knocked down 0.5-1.0 places.  One of the flaws in reading opinions, is that they are based on a single sample or maybe a sample size of 3-4...and if conditions are overly good of bad during those samples...  If 50 raters all see a course in bad condition and downgrade it 25-50 places, there "panel" will knock it down about 37 places...as it should.


I am NOT saying reading well knowledge people's opinions does not have value...but there are flaws in the method as well.  No rating system is perfect.  I have played everything course ever tasted in a Top 100 by 10 different sources...a total of 41 listings and 318 courses.  But even if I was the "perfect, objective knowledgable rater (and I am the first to say that I am not), my ratings are flawed because I have played a good number only 1x, and some in in a long time.


One flaw with GD's system IMO is that it does not give more "weight" to more recent evaluations.  It counts all ratings over the past 8 years and a rating submitted in 2009 will count as much as one submitted 2 months ago.


Paul

[JESII]

Paul - my wish is simply based in knowing the the ranking are flawed. It's all opinion. Why not include the warts and perhaps some commentary around what that person saw in a particular course?


The lists, as currently accumulated, are trying to reach an unattainable place...the truth.

[Jeff_Brauer]

Jim,


for the best new, GD used to send the comments out to panelists and architects, and probably owners, too. Probably not feasible to send them to every subscriber....but they could maybe be put on a special section on the website, who knows.  That said, it is probably a real small subset of people, most of whom post here, that would probably be interested in the detail, or overwhelmingly critical of the whole process.


I think we all agree we can find flaws with anything (its what we do) but Paul makes a good case that they recognize problems and keep trying to improve the system.  We also all agree that getting one numerical ranking that everyone agrees is definitive is impossible, but of course, GD and the others don't care.  If it was so definitive, there would be no reason to publish the rankings next year.


But again, the OP was to define shot values, not debate ranking systems.

[Tom_Doak]

One flaw with GD's system IMO is that it does not give more "weight" to more recent evaluations.  It counts all ratings over the past 8 years and a rating submitted in 2009 will count as much as one submitted 2 months ago.

Paul:


If conditioning is the main point of the ranking then I agree with what you posted above.


But if conditioning is not the main point, then having 50 panelists [including 20 who may not be all that perceptive about golf architecture] instead of a couple of really perceptive ones is a negative.


The same goes for "weighting" the scores as you propose -- it might make sense for conditioning but not so much for design.

[Jeff_Brauer]

My only points in my long posts above were to further flesh out the Golf Digest version to more detail, based on hundreds of better players views.  Perhaps still not a valid sample size.  For that matter, its an attempt to flesh out Tom Doak's definition of shot values, a shot/hole/course (he thinks whole course, actually) that "makes a golfer think first and then execute, if he or she wants to score well."  He and other intuitive supporters think its better to leave it there and to someone's individual perceptions.  It seems equally natural that a Brad Klein or Ron Whitten, in trying to give guidance to hundreds to carry out an assigned task, try to formalize the criteria much more.

One would not have to give guidance to those hundreds if one believed they all had a good understanding of the subject.

Should we just abolish public schools, then, because those kids should somehow just have a good understanding of the subject?


Any perpetrator of a system, whether you, Whitten, or Klein, has an idea of how voters should vote.  You either pick long time cronies who you already know will vote to your liking, or you educate them in the system. (and hopefully, the general art of golf course design evaluation)  Not sure how evaluators are picked, but recall you sort of hand picked (like the ASGCA President every year) but I think the magazines do get more applications they need to vet some, and then educate some.


Not sure, but I don't see a problem with continuing education.


As to weighting, there is no perfect system.  If the editor realized a course had been changed substantially, he might be able to throw out older scores as obsolete, perhaps.  And, there is always the possibility that the one new ranking was one of the outliers, and there would have to be a judgement call whether that was because of great improvement or slide, or the particular voter had an agenda or bad eye......The response would be even more detail, or throwing out the new version until it had the minimum number of voters to see it, which I think they have done.


Again, there will be a flaw in any system you can think of.  But in the end, its okay because the rankings aren't as important as they seem from discussion here.......

[Tom_Doak]

Any perpetrator of a system, whether you, Whitten, or Klein, has an idea of how voters should vote.  You either pick long time cronies who you already know will vote to your liking, or you educate them in the system. (and hopefully, the general art of golf course design evaluation)  Not sure how evaluators are picked, but recall you sort of hand picked (like the ASGCA President every year) but I think the magazines do get more applications they need to vet some, and then educate some.

Name my cronies, please, or take that back.


When I was helping with the GOLF panel, I would get together with the editor every two years, and suggest some people from the golf business to be included.  Usually it was people I'd never met.  I was 25-30 years old and didn't have too many "cronies" at that point ... not sure I do today, for that matter. 


When we'd get letters from people who were volunteering themselves as potential panelists, my procedure was to send them a ballot and ask them to mark down all the courses they'd seen, without grading them ... because I didn't want to be accused of cherry-picking the committee.  The goal was to find the people who had seen the most courses, and especially those who had seen courses in areas where we had trouble getting enough votes [Europe, Asia, Africa, etc.].


The only "education" we ever did when I was involved was to try and encourage raters to get to certain courses where we didn't have enough votes.  Unfortunately they do much more than that now, which has encouraged groupthink and opened the door to outside influence.

[Sean_A]

It's one thing to knock a course down a bit because the conditions are soft and sloppy ... I have no problem with that.  But it seems like you're going to knock it down MORE if it was designed well and the conditions are letting it down, as opposed to somebody building a course that can only be played when the conditions are sloppy.  And the latter is just a bad design, so I don't understand why you'd reward it.

Sean:


I thought my post was pretty clear.  I've never said conditioning should count for nothing, but for me it has to be exceptionally good or bad to make a difference.


One of the problems with rankings and lists, instead of reviews, is that you don't have any room to be clear about why you've come to your conclusion.  In The Confidential Guide, if conditioning has impacted our ranking, we have pretty much always said why ... Portrush is one example in the most recent book. 


But if you knock a course down 25 or 50 places in a ranking because of conditioning, you're making a big assumption that it's always going to be that way, which may or may not be true.  For example, your comment implies that if you'd played The Renaissance Club one year earlier, you would have rated it higher.  That's fine ... but how is someone reading your list going to know which year you played it, or what your objection was?  I have no idea if they're still stuck on the narrow fairways they had for the Renaissance Cup, though I'd bet that sooner or later the members will revolt against them.

Thats fair enough Tom.  Although, if I decide to use conditioning/presentation as factor in the rating I have no idea how many places the course is moved.  It might be 1 or 20 places...that is immaterial because the placings are largely irrelavant. 


Jeff


I gotta say Brad Klein has not said one word to me about how I should rate courses.  The only feedback I have received is an invitation to rate courses for another year.  I don't know if this is good or bad, but I appreciate the hands off approach.


Ciao

[Jeff_Brauer]

Sean,


I have participated in one of those raters education sessions, actually one each for GW and GD.  I wasn't invited back, so I don't know how to take that, either!


I doubt there is much more hands on, or hand holding than that.  Just a few general sessions on both the system and a talk from an architect for general entertainment.


Cheers.

[Tags:]

Tags:

[Tags:]

Tags:

Call to undefined function theme_linktree()