How we evaluate courses

[Peter Pallotta]

Just thinking of Don's original post, and the flip-side (maybe) of decades' worth of analysis and evaluation:

on the one hand, across the board the art-craft of gca has never been better: an industry chock-full of passionate, experienced, talented, educated, technically proficient, committed, responsible and practical professionals, with every mechanical and (sufficient) financial resource at their disposal -- and the work itself always good-to-very good-to excellent

but on the other hand: gone is the accidental, the mad-cap, the unintended, the savant, the slowly-evolving, the beginners, the constrained, the tentative and experimental -- with the work itself characterized by untold countless missteps, and by long-ago-plowed-under failures, but also by several instances of totally unique and enduring greatness (flaws and all)

Ah, well, maybe it's inevitable and in the natural order of things, i.e. you win some, you lose some.
Of course, maybe it's nothing of the sort - just a romantic's rendering of history via a false narrative

But many others (almost infinitely more qualified than me) have praised TOC to the moon while simultaneously recognizing that it would never be built today. Lots of reasons for that, I suppose -- but could one reason be the decades' worth of analysis and evaluation that we all value so highly?
     

[Ian Andrew]

A long time ago my Dad was teaching me about water colours. I struggled with over-painting. He got me to paint less and less until I was down to just a few strokes. He explained that was the essence of what's necessary to add colour to a composition. He then told me I would need decades to figure out how to add four or five more strokes to achieve what I really wanted to paint.


Excellence is not perfection. In fact perfection is actually fairly cold and distant, when presented in most art forms. Excellence lies in those last four or five strokes/choices being charming and engaging rather than misplaced.


Excellence lies in the interaction of multiple elements creating an emotional response in the observer, or in our case the player.
You really think we can mathematically define that ... I don't.

[George Pazin]

A long time ago my Dad was teaching me about water colours. I struggled with over-painting. He got me to paint less and less until I was down to just a few strokes. He explained that was the essence of what's necessary to add colour to a composition. He then told me I would need decades to figure out how to add four or five more strokes to achieve what I really wanted to paint.


Excellence is not perfection. In fact perfection is actually fairly cold and distant, when presented in most art forms. Excellence lies in those last four or five strokes/choices being charming and engaging rather than misplaced.


Excellence lies in the interaction of multiple elements creating an emotional response in the observer, or in our case the player.
You really think we can mathematically define that ... I don't.

Fantastic post, best on the thread, imho.


I think one problem most people, especially smart people, have is a tendency to overthink or over-analyze everything. More is always better, seemingly.


My own personal standard for golf courses is simple: do they ask interesting questions of the golfer? If the question is merely "what's my yardage to this or that?", I don't find that particularly compelling. The really special courses ask interesting questions on almost every shot (which also means that they accommodate most every shot as well).

[Ulrich Mayring]

Well, if golf courses are to be judged on a purely emotional basis, then what sense does it make to rank them at all? Emotions are highly subjective and half of them aren't down to the golf course, but to personal dispositions on the day of play.

And if you answer: "well, it makes no sense", then why are you taking part in this thread?

As an aside, I have yet to see anyone writing course reviews without using numbers to establish some kind of ranking

Ulrich

[Ian Andrew]

I didn’t say don’t rank them, but stop trying to quantify it like it’s science. It’s art.


When I go to a museum - something I do enjoy doing when I travel - I don’t try to apply science to why I connect with a painting. I just do.


But I may look at how the composition was created or find out more on the subject matter to deepen my connection. Or just learn something.


I still like lists because they possess ideas on what to see. It provided me with a great place to begin learning.

[Sean_A]

Well, if golf courses are to be judged on a purely emotional basis...
Ulrich

I don't think this is an accurate representation.  Yes, how a course makes one feel is important, but also how it makes one think is at least as important. 

At the end of the day, one will do what one will do.  A formula/math based approach to decode how you feel and think is what makes you comfortable.  I notice you seem to focus on quirk, shot values, scenery and flow.  Three of those categories are not something I would personally lean heavily on in making a determination about course quality.  More importantly, all the criteria are subjectively important for you.  At some point, it is worthwhile recognizing that to each is own is perfectly fine and that because someone can back up rankings with numbers doesn't mean the system is more valid or better.  The numbers are simply a function of you feel and think.   

Ciao

[Ally Mcintosh]

I think Sean is correct.


I liked Ian's post but don't agree with it 100%. Whilst it is primarily art, there is an element of science also.


I do believe there is some objectivity in evaluating courses. Despite thinking that it's first and foremost how a course makes you feel (and think) on a primal level that determines how good it is.

[Tom_Doak]

I liked Ian's post but don't agree with it 100%. Whilst it is primarily art, there is an element of science also.


I do believe there is some objectivity in evaluating courses. Despite thinking that it's first and foremost how a course makes you feel (and think) on a primal level that determines how good it is.

You're halfway there, then.  You just have to let go of the side of the pool now   Trust me on this, your work will get much better once you stop trying to rationalize it.

[Peter Pallotta]

I’ve often thought about this:

In an art-craft that isn’t wholly improvised, how much planning and rationalizing is necessary and beneficial, and how much of it is actually diminishing and counter productive (especially if you’re not playing it safe or satisfied with the merely good)?

In other words: when do you let go of the side of the pool? Too early and you (likely) drown, though on the other hand you might set a record; too late, and you’ll (likely) survive to swim another day, but you’ll never stand out from the crowd or on the top step of the podium.

Reminds me of the long departed hierarchy of values thread. Not everyone wants or needs most of all to stand on that top step; but if that’s your number one goal it sure does seem like a leap of faith is called for.

Watched an excellent documentary on John Coltrane the other night. He was making terrific music and an excellent living with giants like Miles and Monk - but that wasn’t his goal. And so he leaped, and for him it was literally a leap of faith. And can you believe it? “A Love Supreme” actually became a revered (not too surprising) and best selling (very surprising) album!

I think there’s a hunger out there for true, real, brave and inspiring greatness. But not enough of us ever take the leap.

[Ally Mcintosh]

Perhaps you boys are defining art and science differently to me. Please don't confuse letting go of the side of the pool (i.e. being brave) with the belief that GCA is all art.


I design by feel (art). Occasionally, it does me well to make sure that my eye isn't deceiving me by running a few checks (science). Tom, I suspect you do exactly the same.


As for evaluating courses, I do this by how they make me feel on an emotional level. But an element of objectivity kicks in based on certain golfing attributes that the course may have. You have to know what elements help to make a good golf course in order to constructively evaluate it. Otherwise everyone's opinion should hold the same weight. And The K-Club would be Ireland's No.1 course.

[Tom_Doak]

Ally:


If your definition of science is whether I check greens with a transit, yes I do; it's amazing to see how even the best shapers can be fooled by their own eyes.  And at some point I do a scorecard so I will know how short a course we are building in case the client has questions.


But calling any of the conventional wisdom about golf design "science" is demeaning to actual science.  Many golf course architects have had their own theories and formulas, but none have ever been mathematically proven true.

[Ally Mcintosh]

Ally:


If your definition of science is whether I check greens with a transit, yes I do; it's amazing to see how even the best shapers can be fooled by their own eyes.  And at some point I do a scorecard so I will know how short a course we are building in case the client has questions.


But calling any of the conventional wisdom about golf design "science" is demeaning to actual science.  Many golf course architects have had their own theories and formulas, but none have ever been mathematically proven true.

Valid point. My definition of science was probably wrong or maybe just flippant.


I just don't want people to think that everything is creative and that there is no element of engineering involved. If I have a criticism of myself, I spend too much time on the former and sometime too little on the latter.

[Ulrich Mayring]

Designing/building golf courses is a craft and it does have an element of art, no doubt. But evaluating / ranking golf courses? That's a different beast altogether.

I don't think that any system is superior to another, because it is based on numbers or has the "right" categories. But I do believe there is a value in committing yourself to something. Be that a scale from 1 to 10 or a fixed set of categories, but there should be something to make your work comparable, to allow others to judge where you are coming from and hold you accountable.

Those who believe in a primarily feel-based approach should not forget that emotions are a positive thing only 50% of the time. The other half is you hating a course or getting bored to death by it and are you really going to put those words out there? I believe criticism is important, but you should do it responsibly and give a factual appraisal that adheres to some kind of professional or personal standard. And that standard should be out there for others to look at.

And don't worry, no matter how formal you try to be, emotions will still influence you more than enough

Ulrich

[Sean_A]

Ulrich

Don't confuse a number system with facts. If you want to critique a course you hit the highs and lows. There is no need to assign numbers using a subjective formula and that in no way makes a critique more valid. If anything I would say numbers are a scam because they are used in a subjective manner that at least on the surface is trying to pass for objectivity.

Ciao

[Peter Pallotta]

Coincidental to Sean's post, I'd just been thinking about the value (and relevance to this thread) of his own, personal rating system, ie
3*  Don't miss for any reason
2*  Plan a trip around this course
1*  Worth the expense of an overnight stay
R    Worth a significant day trip (no more driving than it takes to play and have drinks)
r    A good fall back on course/trip filler
NR Not recommended

This approach is more telling (it seems to me) and more useful (as a guide to someone like me) than the traditional numerically-based systems (e.g. Top 100) where each course gets a score and then is ranked accordingly -- and where the difference in the scores between the 1st and, say, 15th courses is very often equal to or even greater than the difference between *all* the other courses on the list combined.
(E.g. if No. 1 scores a 9.4. and No 15 a 7.8. -- a difference of 1.6 points -- it sure seems that between No. 16 and No 50 you'll find the same 1.6 point/score difference.)

Now, this is not to disparage those lists or even the 'criteria' involved in rating courses (we've already done that endlessly and ad nauseam -- and I very much enjoyed Ben's Michigan list and Brian's Ohio list, especially the write-ups). Instead it's to note:

a) that, while like Sean's approach these kinds of lists do very well at highlighting the best of the best (about 10%), unlike Sean's list they do a relatively poor job of guiding a would-be traveler on deciding which course among the remaining 90% (presumably all good-to-very-good courses) is appreciably better or more interesting than any other; and to note
b) that this difference between the two approaches suggests something important about the topic at hand, i.e. how we evaluate golf courses

What it suggests is that the decades-long approach to course evaluation (ie one that consciously or not uses the same basic approach/set of expectations for every single course being evaluated) has led to the vast majority of (even very good) golf courses being virtually indistinguishable in terms of quality and interest.   

I wonder what the potential impact might've been on gca if, 60 years ago, the magazines and the rest of us had adopted the Arble Scale, which seems to preference *uniqueness* and *exceptional quality* above all else.  There wouldn't have been much place to hide, or even much room for nuance for any new course that was built:

Golfers might plan an entire trip (and pay for an overnight stay) simply to play course X; or they might make a long drive to play it; or they might skip it all together! That would've gotten one’s attention...

[Sean_A]

Pietro

Thank you, the approach is meant to serve as a guide for travellers so it is good to get affirmative feedback.  My approach is a take on Doak's and heavily leans on the Michelin Guide...obviously. I don't think there is nearly enough distinction between courses to warrant a 1-10 scale....though I expect I am in the minority on this issue.  Perhaps the most important aspect of the approach is that I don't make a final judgement on the quality of a course.  This is in the main due to my belief that course quality is but one of at least a few reasons why people travel to play golf or why they choose to play certain courses.  From personal experience, if a course is of a certain minimal standard which I admit is a purely subjective matter, I can like it just as much as the best courses in the world.  In other words, course quality is not enough to guarantee that I will like it.  The enjoyment of a game and course revolves around so much other than the quality of a course that I never felt the need to worry about a best list...even if I could figure out how to go about it in a satisfactory manner. 

Ciao

[George Pazin]

If anything I would say numbers are a scam because they are used in a subjective manner that at least on the surface is trying to pass for objectivity.

Ciao

I love this thought.

[Ulrich Mayring]

Sean,

I don't understand. You are using numbers. Tom Doak is using numbers. Everyone is using numbers. Can you point me to any reviewer that is not using numbers?

And yet you say numbers are a scam?

Ulrich

PS: Apparently you even have categories: "the Arble Scale, which seems to preference *uniqueness* and *exceptional quality* above all else" - at least that is the perceived opinion.

[Sean_A]

Sean,

I don't understand. You are using numbers. Tom Doak is using numbers. Everyone is using numbers. Can you point me to any reviewer that is not using numbers?

And yet you say numbers are a scam?

Ulrich

PS: Apparently you even have categories: "the Arble Scale, which seems to preference *uniqueness* and *exceptional quality* above all else" - at least that is the perceived opinion.

Ulrich

No, I don't use numbers in the same way you do....so far as I can tell.  My numbers are merely labels for convenience.  I don't run through a series of categories valued 1-10 (or whatever) with weighted values etc etc to determine a final formula number grade.  Besides, I am not ranking courses by quality so going through a hullabaloo such as that would be silly.  I don't give a fig if Muirfield is better than North Berwick.  I am offering my opinion to golfers willing to travel as to which courses they should consider playing.

Ciao

[Ulrich Mayring]

Well, we are making progress here

So you are in fact using numbers, but the debate is on the correct way of using numbers.

You seem to prefer to assign one number arbitrarily, whereas I assign them arbitrarily in four categories and take the average of those. So the world of difference between our systems appears to be whether to assign one number arbitrarily or four and whether to use a scale of 1-6 or 1-10.

Ulrich

[Sean_A]

Well, we are making progress here

So you are in fact using numbers, but the debate is on the correct way of using numbers.

You seem to prefer to assign one number arbitrarily, whereas I assign them arbitrarily in four categories and take the average of those. So the world of difference between our systems appears to be whether to assign one number arbitrarily or four and whether to use a scale of 1-6 or 1-10.

Ulrich

No, my numbers are a short cut.  There is no significance to the label in and of itself.  It is simply easier to write r rather than a good fall back course/trip filler.  So far as I can tell, Doak does the same thing.  You on the other hand use weighted categories whereby the numbers are indications of quality and are used to determine a final number quality score.  I think you know there is quite a difference 

Ciao

[John Kirk]

I'll bump this back to the top.

In the persistent search to find thoughtful ways to describe how golf courses should be evaluated, I will add the following thoughts.  I have not played Cypress or Pine Valley, but I have played approximately one-third of the most coveted courses in the U.S.

-  A great golf course should give the player a sense of place.  Whether it is the oak dotted coastal plains of south Texas, or the mixed deciduous forests of coastal southern Oregon, a great golf course should remind the player of where they are.  In the case of St. Andrews, Scotland, the sense of place is largely defined by the town itself.

-  With rare exceptions, a great golf course should have short, logical green to tee walks, and provide a compelling walk through the golfing park.

-  A great golf course provides a grand variety of shots that stimulate the player's senses in a positive fashion.  The shots offered consistently evoke excitement and anticipation, as opposed to evoking dread or fear.


Regarding the "math" discussion, any numbers used to evaluate a course are an approximation of an analog scale.  It's not really mathematical at all.  And finally, I know a few people here who are math whizzes, the kind of people who probably (or did) score 800 on their math SAT when they were kids.  Some or all of them have been, or are still course raters.  All of them treat course rating as an analog, intuitive exercise.  None of them are category driven, just a general sense of how good it is.

[Ulrich Mayring]

Sean,

I honestly don't see the difference between what you describe as your way of rating courses and what I do. We are both using numbers as shortcuts for a narrative and we are both saying 3 is better than 2.

We are also both rating quality, only we may have different definitions for quality in golf courses. I haven't heard yours, but it appears to have something to do with trip planning.

Personally, I don't believe in second-guessing people which number would be worth a detour for them or even worth planning an entire trip around and which number they would see as "not recommended". As a player I would ask the rater this: not recommended for who and in which circumstances? I occasionally find myself playing courses that suck, but if I am itching to play golf that day and those are the courses available that day, then I will play those courses that suck and I will have fun.

I'm thinking that for me an 8 is better than a 5, but I try not to make any blanket recommendations. I've seen players hating courses that I praised and vice versa.

Ulrich

[Mark_Fine]

With all due respect, it always boils down to numbers!  Whether you are judging paintings in a museum, deciding which is your favorite ski resort, discussing the best restaurants in town, talking about the prettiest girl at the dance, determining who is the class valedictorian, or deciding which golf courses are the best in Scotland - it all comes down to numbers!  Do we walk around the Louvre in Paris saying that painting is a 7 and that one is a 9?  No, but if someone asked us on the way out which ones were our favorites, many of us would have an answer.  If asked why we liked them so much we would hopefully be able to articulate the reasons for our preferences.  Stating what our favorite golf courses are is no different. 


I never really understood why there is so much resistance to the concept of numbers?  It is what it is.  Human nature loves to rank things and put things in some kind of order and that order is all determined by certain criteria and preferences.  And in the process we give values to those criteria and preferences (whether we want to admit it or not).  We may or may not write down a numerical value for each criteria but at the end of the day, we make a determination as to our favorites (our ranking) based on those combined values. 


When Peter says he uses a 3-r scale, it just another numerical scale.  If he was pressed to narrow his scale down even further (e.g. what are the best of the must see courses that he calls 3s?) he would be forced to figure out a way to determine what they are.  If he said he couldn’t then some would argue he is just not as knowledgable about his subject matter as he needs to be.  I don't know if Peter is married or not but if he is not, maybe it is because he was never able to narrow that list down either     


Ran once told me that he could tell me why his 42nd ranked course is better than his 43rd ranked course.  Frankly, I would struggle to do that.  I actually like the Doak Scale and I can get down to what is an 8.5 vs an 8 but that is where if I am honest with myself I get tapped out.  After that level of scrutiny, it is more flip a coin as to which 8.5 is better than another 8.5. 


There is nothing wrong with numbers.  It is more about "what formulates those numbers" that one ends up with.
Mark

[Sean_A]

Sean,

I honestly don't see the difference between what you describe as your way of rating courses and what I do. We are both using numbers as shortcuts for a narrative and we are both saying 3 is better than 2.

We are also both rating quality, only we may have different definitions for quality in golf courses. I haven't heard yours, but it appears to have something to do with trip planning.

Personally, I don't believe in second-guessing people which number would be worth a detour for them or even worth planning an entire trip around and which number they would see as "not recommended". As a player I would ask the rater this: not recommended for who and in which circumstances? I occasionally find myself playing courses that suck, but if I am itching to play golf that day and those are the courses available that day, then I will play those courses that suck and I will have fun.

I'm thinking that for me an 8 is better than a 5, but I try not to make any blanket recommendations. I've seen players hating courses that I praised and vice versa.

Ulrich

Ulrich

I don't have any idea of what your narrative is for an 8 or 5.  My system offers a fairly clear narrative as to whether or not I think a course is worth the effort, time and money to play.  Additionally, I give a modest picture tour with brief descriptions to enable the reader to judge if my summary is accurate or worthwhile.  I accept that we all can't agree. 

I am not rating quality regardless of how our views may differ.  If I were focusing on a quality list my main criteria would be

-routing: quality of the walk, views, green sites and natural features

-the site: terrain and quality of soil/grasses

-the greens: firm and true or otherwise appropriate to the climate and design

-man-made features

I wouldn't bother with the house, cost, likely pace pf play, history etc...important aspects for many people when it comes to choosing where to play...but a quality list isn't about that.

My list of best courses would look very different to my favourites and different again to the recommendations.  As I said previously, I don't care how good a course is once a certain level of quality is achieved. 

My recomemendations are for the general population of golfers who are willing to travel...the scale make this obvious.  This population can choose to take or leave the recommendations.  We all play crap courses here and there, but do you recommend others to play those crap courses or rate the courses highly because you had fun?

We shall have to agree to disagree about the use of numbers. I emphatically do not use a number system or formula.  Trying to do that when evaluating what is actually in the ground instead of what I want to be in the ground does not, in my experience, work very well.  Or maybe a course is excpetional in one area and poor in another.  I may decide the exceptional far outweights the poor and recommend the course higher...or conversely decide the poor aspects outweigh the excpetional elements.  Each course is different and treated as unique.

Mark - I don't really care if you use numbers.  As you say, it seems to be human nature.   

Ciao