Four Biases in Course Evaluation

[John Kirk]

Next month I will have been a member of GCA for five years.  I've observed how people evaluate courses, and see at least three distinct patterns.  Here are four potential biases when we evaluate a golf course:

1.  Regional Bias
2.  Multiple Play Bias

Both how close a course is to home, and how many times a course is played, are strongly correlated with one's perception of a course's quality.  Regular GCA members often cite a course in their state or metropolitan area as the best.  In many cases, this may be a matter of repeat play bias.  It is clear that a great golf course gets better with repeat play, as the golfer learns the subtleties and possibilities for each great golf hole, which is only revealed with multiple plays.  There have been a few instances where I have been "blown away" or "floored" by my first round at a course (Riviera, Merion, but not Crystal Downs or Winged Foot), but these are far and few between, and can just as easily be attributed to my mood on that given day.  I have now played Crystal Downs four times, and each new visit reveals something new and exciting.  Even Pacific Dunes required a second play before I was ready to fully embrace it.

I find it hard to accept that even a veteran golf rater can accurately assess a course with one round.  The trained observer can look for certain features, and a course's beauty should factor into evaluation, but how can you really know how it plays unless you play it?  An evaluation by a large panel of raters, many of whom base their rating on one attempt, is a reasonable substitute.

It is heartening to see GCA members regularly cite courses near their home as among the best.  There's a love for one's home, and a belief that the grass is greener in their own backyard.

3.  Style Bias

Course evaluators tend to appreciate a style of golf course they are familiar with.  A negative statement is closer to the truth.  Evaluators tend to dislike a course style they are unfamiliar with.  I see four primary styles of golf:

A.  Parkland courses
B.  Linksland courses
C.  Desert mountain courses
D.  Southern resort courses

Beyond those, there are courses which don't neatly fall into any one category, which makes places like Prairie Dunes or Cypress Point unique and special.  But if you take a golfer, who only plays parkland courses, out to the windswept links or a hilly mountain layout, he is likely to be uncomfortable.  The new challenges require different skills to navigate the layout.

However, my experience is different.  I first experienced linksland golf in 1998, on a visit to Scotland with my father.  My rounds at Royal Dornoch, followed by regular visits to Bandon Dunes, showed me that a firm, undulating course, exposed to the wind, created a more complex challenge.  It was love at first sight, as my mathematical mind saw the added complexity of controlling one's ball trajectory, not to mention the mental rigor of a five foot putt in a high wind.  Bandon Dunes changed my life.

Golfers are often repulsed when introduced to some aspects of golf design that we at GolfClubAtlas covet the most: the undulating green, the occasional blind shot, and the uneven lie.  I believe most golfers would begin to embrace these concepts with repeat exposure.

Parkland courses are a special case.  My first years playing golf were spent mostly at the Stanford University course, a wide, hilly course on oak-studded foothills, not a typical parkland design.  I did not begin golf with a positive bias to this type of course.  Older parkland courses generally share the following features:

A.  Softer playing surfaces, with dominant, mature poa annua.
B.  Large numbers of specimen (and often, non-native) trees lining the fairways.
C.  Ovoid greens, lower in the front.
D.  Sand and water hazards in traditional locations, some strategic and some penal.

I find it very hard to distinguish one parkland design from the next, and perceive parkland golf courses as a "dime a dozen", especially those built during the Golden Age.  Up here in Portland, Oregon, the best parkland courses, such as Eugene, Portland, Royal Oaks, and Waverley, are all about the same, and require the same soft approaches and punch out recoveries.  This takes us to our final bias.

4.  Architectural Pedigree Bias

I find it nearly impossible to distinguish the work of top notch Golden Age architects.  Most eastern parkland courses are indistinguishable to me.  In my five years at the site, I have never seen the different philosophies clearly explained to me, if they truly exist.  I've picked up little tidbits along the way, like Ross's affinity to route the fairway in the valley and the green on a knoll, but I could never tell you by viewing ten pictures who designed a course.  The distinguishing characteristics may have disappeared, due to maintenance practices and the work of overzealous greens committees, leading to their homogenous appearance.  A clear explanation of the differing philosophies, especially between Tillinghast, Ross and Flynn, would be a fine addition.

As a lifelong west coast resident, I think I can tell a MacKenzie or a Thomas/Bell from other designs, but that may be a recognition of the terrain itself, and not the architect.  It would be easy to see the Sonoma Golf Club, with the oak dotted hills, and mistake the Willie Watson design for another west coast architect.  Is the home state bias in play here, or is it like different dialects of the English language, with a familiarity of the local slang?

Finally, there is much discussion at GolfClubAtlas about the merits of each architect.  However, I do not see a strong predisposition to applaud our favorites, or criticize others, before personally evaluating the work.  As a group we have a open mind, and though many of us are loathe to offer criticism, the lack of support for new high profile designs by our favorites is an implied lack of endorsement.  Our members gush over Friar's Head, while less support is given to Coore and Crenshaw's ambitious Colorado Golf Club.  Some of Tom Fazio's efforts are dismissed as pedestrian, while newer projects like Pronghorn, Martis Camp and the Madison Club are given impressive reviews.  I recently said I disliked Rees Jones's Sandpines, but that does not mean I won't enjoy Ocean Forest or Atlantic immensely.  Robert Trent Jones Jr.'s Chambers Bay was universally embraced by the community.  I say we are an open minded group, with little preconception.  We want to enjoy the new golf experience, and start with a positive attitude.

[John Kirk]

I'll be back in a few hours.

[Tom Huckaby]

John - that's all good stuff - but can you clarify what you mean by "evaluation" of courses?  Is it how much we like them?  What we say about them on here?  Which ones we personally consider the best, or not?  I think that's what you are getting at but I can't tell for sure.

If you mean evaluation in terms of magazine ratings, I think it's a different equation.  But before we battle that again - and good lord do we not need to do that - I figure it's best to maybe head this off at the pass.

TH

[Anthony Fowler]

Great points John.  I have seen these biases most notably in myself.  I'll add one more:

5. Myopic Bias

People remember best the courses that they have most recently played or have most recently seen hosting  a major tournament on television.  This may cause them to bias their favor towards these courses.  I find myself doing this frequently.  Hopefully I will improve in my evaluations as I become more aware of my biases.



John, I agree that the four biases you have mentioned are real, but it is impossible to determine the magnitude of these biases because of the possibility of reverse causation.  I will explain.

1. Regional Bias: Perhaps instead of people liking courses because they are close to them, people actually move to the regions where they like the golf courses.

2. Multiple Play Bias: People go back to the courses they like.  They don't like them more because they have played them a lot, they play them more because they like them a lot.

3. Style Bias: People play the styles that they like, they don't just like the styles they happen to have played.

4. Architectural Pedigree Bias: People speak highly of architects whose courses they like.  When they speak highly of a course and its architect, it seems their view of the architect affects their view of the course, but it could be that their view of the course affects their view of the architect.

I think in all cases, there is some of the bias that John described.  However, the problem is not as drastic as it seems, because these reverse causation stories are also true in many cases.

[John Kirk]

I guess I'm not gone yet...soon.

Tom H.,

Yes, that's what I mean.  I am trying to determine inherent bias in how good the golfer thinks the golf course is.  I think it directly applies to magazine rater evaluation, but it's not my objective to discuss that specifically.

Anthony,

Wonderful response.  Thanks for that, and I may riff on your points later.

[Tom Huckaby]

John:

Excellent.

I think you are pretty darn right on re all of this.

I just also see none of this as a bad thing. I doubt you meant to suggest that either... but anyway, how people look at golf courses, of course they are going to have biases and preconceptions and the like.  You just described how it works with golf courses darn near perfectly... at least for those very into "architecture" as most here are.

I would just add some other biases, however:

COST
and that works both ways... high prices can bias upward, but bang for the buck can as well. 

PUBLIC/PRIVATE
not so much in here, but definitely in the real world... privates are seen as superior to publics, in general.

and then a very key one for this group.....

REMOTENESS
if one has to travel very painfully far to see a place, one doesn't want to admit it was anything short of a great use of his time.  THus remote courses - particularly those that are difficult to access - start out ahead of the game.  It doesn't END the game, but they do start out ahead.

TH

[Carl Nichols]

I suppose it depends on what one is evaluating -- e.g., the overall experience vs. pure design/architecture.  If you're talking about the latter, I think there are biases relating to conditioning, amenities, and accessibility, that is, certain people allow their assessment of a course's pure design/architecture to be affected by the conditioning, the quality of the amenities (clubhouse, etc.) and how exclusive the place is.  I'm almost certainly guilty of this.  

[Tom Huckaby]

I suppose it depends on what one is evaluating -- e.g., the overall experience vs. pure design/architecture.  If you're talking about the latter, I think there are biases relating to conditioning, amenities, and accessibility, that is, certain people allow their assessment of a course's pure design/architecture to be affected by the conditioning, the quality of the amenities (clubhouse, etc.) and how exclusive the place is.  I'm almost certainly guilty of this.  

Carl - please do not say you are "guilty" of that - you are a golfer, correct?  Not in the architecture business?  If so, then of course it's best for you to evaluate the entire experience - isn't that what matters to you most?  Then if you choose to try to evaluate the "architecture", you do your best, but you know that you can't know all that went into it, so offer such only with an ocean of salt.

Or at least that's what I do, as a golfer.  Let those in the business evaluate architecture... stick to evaluating golf courses.

TH

[Kalen Braley]

John,

Really interesting topic!!

I think Tom's Remoteness theory is interesting.  When you throw in remote, exlcusive, and relativly new, it can almost be an everest effect of sorts where playing the course becomes a conquest not done by many others.

In my experience, I think the biggest biases comes from varying styles.  With all the various styles already mentioned and including some additional ones like prarie-land, ocean-based, eye-candy, immaculate conditioning, there really is a wide spectrum for variance out there.

Perhaps, its just best to evaluate the best in each category and not even try to cross-pollenate a rankings list.  I mean really, how could a Pine Valley and Royal County Down even begin to be compared?

[Carl Nichols]

I suppose it depends on what one is evaluating -- e.g., the overall experience vs. pure design/architecture.  If you're talking about the latter, I think there are biases relating to conditioning, amenities, and accessibility, that is, certain people allow their assessment of a course's pure design/architecture to be affected by the conditioning, the quality of the amenities (clubhouse, etc.) and how exclusive the place is.  I'm almost certainly guilty of this.  

Carl - please do not say you are "guilty" of that - you are a golfer, correct?  Not in the architecture business?  If so, then of course it's best for you to evaluate the entire experience - isn't that what matters to you most?  Then if you choose to try to evaluate the "architecture", you do your best, but you know that you can't know all that went into it, so offer such only with an ocean of salt.

Or at least that's what I do, as a golfer.  Let those in the business evaluate architecture... stick to evaluating golf courses.

TH

Tom:
No real disagreement here.  All I meant to say is that when I venture into the world of offering an opinion regarding a course's pure architecture/design -- not how it was built, etc., since I know nothing about that, but the result of that process -- I am probably influenced by other matters that aren't purely related to architecture/design, like conditioning/exclusivity/amenities.  I suspect others are as well, even if they purport to separate those matters out (like when they vote on various categories for the GD ratings).  Put more concretely, I suspect that I'd evaluate the exact same par 4 differently, in terms of its architectural/design merit, if it's behind the gates at Bel Air than if it's at Simsbury Farms. 

[Tom Huckaby]

Carl:

I have zero against any of that.  I just take it a bit farther than you do, in that given I know zero about what went into the design, how can I effectively evaluate the merits of it?  So I don't even try.

I stick to evaluting the golf course and how much joy it gives to play - which not only seems logical to me but also is really all I care about - and in so doing, how can I not take into account things like views, conditions, events that have taken place there giving the place a cool feel, etc?  It's really what you say you do also - I just find zero wrong with it, and in fact think it's the best way to evaluate golf courses for golfers.  Thus your apologetic words seem unnecessary to me.

Now as for how it's done for Golf Digest - careful, I am a panelist for them also.  John made it clear that's not the issue here.  But rest assured it's pretty darn simple to give a 1-10 rating for very specific, clearly defined criteria - and that is what we do.

TH

[George Pazin]

Nothing about player bias? I.e. How someone played the course and how it relates to his game specifically? I've always felt this was one of the single biggest biases.

[Tom Huckaby]

Nothing about player bias? I.e. How someone played the course and how it relates to his game specifically? I've always felt this was one of the single biggest biases.

I don't doubt such exists; it is quite a bit easier to feel happy about a course after playing well there.  However, I'd also say this is the easiest bias to ignore/put aside, as it is so obvious. 

TH

[Carl Nichols]

Tom:

I think we may be using the term "design" in different ways.  I too know very little about what went into building a particular golf course, but I can look at the course as it is presently laid out, how the holes are currently structured, etc. -- that is how I'm using the term "design."  My only point is that when I hear people say "Course X is a great design," that implies an evaluation of *just* the design, independent of other factors, when I think those other factors often influence how people view the "design."   

Also, I didn't mean to suggest I have any negative views about the GD ratings -- among other things, I like seeing how the courses fare relative to each other in different categories. 

[George Pazin]

Nothing about player bias? I.e. How someone played the course and how it relates to his game specifically? I've always felt this was one of the single biggest biases.

I don't doubt such exists; it is quite a bit easier to feel happy about a course after playing well there.  However, I'd also say this is the easiest bias to ignore/put aside, as it is so obvious. 

TH

Such delusion simply shows how ingrained it is with you....

What I meant was something more along the lines of the exchange between Rich Goodale and Tom Doak on Rich's recent "short par 4 made better when lengthened" thread. I think it's almost impossible for 99.9999999% of golfers to look past how he or she interfaces with a course, unless he simply walks it and studies other players.

At the very least, I think it deserves a mention along with the other categories.

[Peter Pallotta]

Just thinking out loud, but how about "The Experienced Bias", which might be one unique to golfers who've played a lot of courses. Since they've seen many of the classic features and variations on a theme and all manner of Par 3s, 4s, and 5s, there may be a tendency to equate -- too quickly, and without leaving room for later discovery -- every new hole they play with one that they've played before...thereby missing some of the fullness of the new experience and the possible uniqueness of the new golf hole. 

Of course, there can be the "Inexperienced Bias" too, whereby a less travelled golfer can miss the "references" to classic golf holes when playing on new hole for the first time, and thus miss some of the (implicit) clues regarding strategy and options

Peter  

[Tom Huckaby]

Tom:

I think we may be using the term "design" in different ways.  I too know very little about what went into building a particular golf course, but I can look at the course as it is presently laid out, how the holes are currently structured, etc. -- that is how I'm using the term "design."  My only point is that when I hear people say "Course X is a great design," that implies an evaluation of *just* the design, independent of other factors, when I think those other factors often influence how people view the "design."   

Also, I didn't mean to suggest I have any negative views about the GD ratings -- among other things, I like seeing how the courses fare relative to each other in different categories. 

I guess this does turn on what one means by "design."  What you describe is the golf course minus its surroundings and what you call "other factors."  I don't see the worth in evaluating that, as what does it matter?  Do those other factors not exist when one is playing?  Does one not feel them?  And then if one tries to evaulate what I guess we'll call "just the golf course", how can one truly evaluate it effectively minus knowledge of what went into building it?  Truly what is one evaluating then?

It makes little sense to me.  But if people want to try to evaluate that, then yes, I see what you are saying - they are even fooling themselves doing that, as the "external factors" do have their way of creeping in.

As they should, I'd say.....

TH

[Tom Huckaby]

Nothing about player bias? I.e. How someone played the course and how it relates to his game specifically? I've always felt this was one of the single biggest biases.

I don't doubt such exists; it is quite a bit easier to feel happy about a course after playing well there.  However, I'd also say this is the easiest bias to ignore/put aside, as it is so obvious. 

TH

Such delusion simply shows how ingrained it is with you....

What I meant was something more along the lines of the exchange between Rich Goodale and Tom Doak on Rich's recent "short par 4 made better when lengthened" thread. I think it's almost impossible for 99.9999999% of golfers to look past how he or she interfaces with a course, unless he simply walks it and studies other players.

At the very least, I think it deserves a mention along with the other categories.

Wow, I'd disagree with that quite strenuously.  I find it pretty darn easy to look past how I interface with a course, and I surely don't need to study other players.  I don't find this to be difficult at all.  It certainly can't be me.  Just imagine hitting shots from other places, with other degrees of success... jeez nearly no one hits the ball the exact same way every round, every shot.  Each golfer has a wealth of experience to draw from.  And if not, well... it also doesn't take a world of imagination to do any of this...

I would agree this deserves a mention; I just don't see it as being one of the biggies.  But perhaps I am the smartest most aware golfer on earth and the rest of you are dumbshits.

 

[Mike Policano]

John,

Excellent post.  As I read your list of biases, I thought of the line "we are prisoners of our own experience".  Often times when you see something new that is different than what you have seen before, it is difficult to initially evaluate it because you are comparing it to what you have seen before.

For example, I brought someone to a course that was brown and proud of it.  On the fourth and final day, my guest said, "You know, I didn't get this course the first day and I was really turned off by the brown.  But after a few days here, I really like this course with its firm fairways and greens." 

I brought another guest to a place with greens inside of greens and pretty undulating.  He commented a few times, where is the windmill?  He has never been to Ireland or Scotland and he belongs to a soft parkland course.  Yet another guest who loves undulating greens smiled childishly as he putted off a green.  He couldn't wait to try the same putt on a different line.

So as one sees more variety, does one become more "biased" because of likes and dislikes or less biased?

[George Pazin]

I would agree this deserves a mention; I just don't see it as being one of the biggies.  But perhaps I am the smartest most aware golfer on earth and the rest of you are dumbshits.

 

I usually preferred to be addressed more formally, as Mr. Dumbshit.

You might think you're looking past your own biases, but that's part of the difficulty. Heck, you think Black Mesa is very playable for high handicappers.

[Tom Huckaby]

I would agree this deserves a mention; I just don't see it as being one of the biggies.  But perhaps I am the smartest most aware golfer on earth and the rest of you are dumbshits.

 

I usually preferred to be addressed more formally, as Mr. Dumbshit.

You might think you're looking past your own biases, but that's part of the difficulty. Heck, you think Black Mesa is very playable for high handicappers.

I don't recall ever saying that about high handicappers at Black Mesa.  What I think I said was that it wasn't awful or anything.  And having hit a few shots there that were worthy of a 30+, I believe it didn't take much imagination.

And therein lies my point.

I know that if I choose to look past my own game when evaluating a golf course - which is darn near always wise - I can pretty easily do so.    You may choose to disagree or not believe me, and that is just fine.  But I know I have done this many times, and as I say, it is not all that difficult.  One just draws on one's experience of shots... or if not, just imagines what might be.

TH

[Kalen Braley]

I will testify that Tom H knows what crappy shots look like.  After all he was subjected to my crappy play during the last KP event and did his best to pretend like he was patient after routinly driving 50-60 yards past me.

As Benham says, he's sneaky long off the tee!!   

[Tom Huckaby]

I will testify that Tom H knows what crappy shots look like.  After all he was subjected to my crappy play during the last KP event and did his best to pretend like he was patient after routinly driving 50-60 yards past me.

As Benham says, he's sneaky long off the tee!!   

Flattery will get you everywhere. 

But seriously, of course it's also not hard to watch the shots of others, the good and the bad.  If I was 50 yards past you Kalen (and I sure don't recall that), well... I am used to being routinely 50 yards behind Benham himself.  So it works all ways.

In any case yes, it's also not hard to just be aware of what others are doing - and this is not any great "study" of other golfers, as mentioned before.

TH

[George Pazin]

I would agree this deserves a mention; I just don't see it as being one of the biggies.  But perhaps I am the smartest most aware golfer on earth and the rest of you are dumbshits.

 

I usually preferred to be addressed more formally, as Mr. Dumbshit.

You might think you're looking past your own biases, but that's part of the difficulty. Heck, you think Black Mesa is very playable for high handicappers.

I don't recall ever saying that about high handicappers at Black Mesa.  What I think I said was that it wasn't awful or anything.  And having hit a few shots there that were worthy of a 30+, I believe it didn't take much imagination.

And therein lies my point.

I know that if I choose to look past my own game when evaluating a golf course - which is darn near always wise - I can pretty easily do so.    You may choose to disagree or not believe me, and that is just fine.  But I know I have done this many times, and as I say, it is not all that difficult.  One just draws on one's experience of shots... or if not, just imagines what might be.

TH

And just how do you know your thoughts are correct, or even objective?

[Tom Huckaby]

And just how can evaluations of a golf course ever be "correct"?  In the end they are opinions.

As for objective, it goes back to it being very simple to draw from past experience of shots, or imagine other extremes of shots, as explained before.

TH

[Tags:]

Tags: