Four Biases in Course Evaluation

[George Pazin]

And just how can evaluations of a golf course ever be "correct"?  In the end they are opinions.

As for objective, it goes back to it being very simple to draw from past experience of shots, or imagine other extremes of shots, as explained before.

TH

So if they are opinions, how can you overlook the biases of the opinion holder? Why even mention the other 4 biases if you are going to ignore the bias of the play itself?

[Tom Huckaby]

And just how can evaluations of a golf course ever be "correct"?  In the end they are opinions.

As for objective, it goes back to it being very simple to draw from past experience of shots, or imagine other extremes of shots, as explained before.

TH

So if they are opinions, how can you overlook the biases of the opinion holder? Why even mention the other 4 biases if you are going to ignore the bias of the play itself?

Who said one ought to ignore it?  I've said it can matter.  I just do find it easy to overcome; in fact the easiest of those so far mentioned.

But we have mucked up John's thread enough.  I get it, you disagree with me.  So that puts our lifetime score at about 99% disagreement, all issues considered.  And here I was hoping to get it to 98....

TH

[John Kirk]

Guys,

Thanks so much for your initial replies.  My initial post, though carefully written, is not intended to be the last word on bias.  I've been thinking about the role of bias in course assessment for some time, and finally sat down and wrote a formal essay on the topic.  Based on the fine input, it may make sense to expand the essay to include many of the additional biases mentioned.  I waited three of four days, trying to think of other factors, but could only come up with the four.

The whole concept of bias is testable in some cases.  For instance, Brad Klein could see how the Golfweek raters rated the courses in their home state, compared to out of state raters.

The whole concept of reverse causation is worth including in a full discussion.  Very insightful.

Thanks again.  I'll be back.

[Tom Huckaby]

John - it is a hell of an interesting topic and I am certain your essay will be insightful with or without our help.

I would just caution on one thing:  be careful basing it too much on how people IN THIS FORUM see things - if in fact you are intending this essay for the real world.  THis is most definitely not the real world, but rather a gathering of zealots (and I mean that endearingly).

The biases of golfers outside this forum will be wholly different....

And since there are so many more of them then the zealots herein, it just seems to me to make sense to write more about them than us.

One example is the "style" bias... real world golfers certainly have their biases about certain types of courses, but very very few give a rat's ass who the architect is or the school of architecture exemplified at the course or anything like that... which of course is damn near paramount in here.  Just be careful to think of it that way.... limit your bias #4 to zealots only.

TH

ps - if the essay is intended just for those in here, then well.... never mind.   

[Andy Troeger]

John,

I'm not sure I agree with the regional bias assertion. As I look through my own lists of evaluations, if anything I tend to be harder on courses in my home states than I am on ones that I see on vacation. Out of 267 courses I've played, 104 of them are in Indiana, but only something like 14 of those in Indiana make my top 100. In my current state of New Mexico, only Black Mesa and Paa-Ko Ridge make my top 50, although Las Campanas Sunset and Pinon Hills come fairly close.

I do certainly agree with the style bias and have had that conversation with a fellow GCA'er recently. Some courses fit certain golfers' eyes better than others. I think its hard to lump courses into only four categories though, to me mountain, canyon, and desert golf can all be very different things. On those types of courses, I tend to really look at how much playable width there is between the native areas, and whether the native areas themselves provide any chance at recovery. I do tend to like mountain and canyon courses with a more neutral opinion toward desert courses. Sometimes I like prairie courses, usually if they have some contour/undulation to them. Really flattish looking courses without some trees usually are not normally aesthetically pleasing to me, I need something to interrupt/change  the view eventually I guess.

[Mike Sweeney]

John,

I would add one of my huge biases formed at least partially from this website is for sand based courses.

Newport
Fishers
Pine Valley
Mountain Lake
Cypress Point (I believe it is sandy based)
Sand Hills

All very different courses, but all sand based.

In reference to the regional bias, I agree. Just played Baltimore CC for the third time, and it keeps growing on me. Blows away many of the NY Tilly courses often mentioned here.

[Sean_A]

John

I think you raise many good points and Anthony also raises excellent counter points.  I can certainly agree with Anthony, in my case anyway, that I tend to go back to places I like and will often do so at the sacrifice of seeing something new.  If I see a course once or twice and am not impressed I generally don't go back.  In this regard, and it is without a doubt superficial, I use the pix posted on this site to at least help guide me where to go and where not to go. 

Which leads me to another of my biases which John raises, style (and it doesn't matter matter to me who designed the course).  There is no question that I like a low key style based on sandy turf which takes full advantage of God given elements and am not terribly interested in seeing desert or mountain golf.  The look is almost always too jarring for me.  A big part of enjoying the architecture for me is in the details.  When I see many desert courses they look as though the detail work was skipped.  That isn't to say I can't be dragged to the once in a while mad dog course - I do like Tobacco Road. 

Leave it to AwsHuckabilly to identify one critical element for me and re-affirm two other issues.  Cost is of great importance to me.  I am happy to throw the sheckles about, but the more I throw, the more I expect.  Unfortunately, it is the rare course that impresses me so I am leary about spending loads of dosh for golf.  I spose I am from the school of there ain't all that much difference between archie quality out there.  Some courses are clearly better than others, but that doesn't mean they are worth the money.  And, as Huck states, the experience is as important as anything. 

I would also agree with Huck that I can absolutely, unequivically separate my game from my opinion of a course.  That helps to explain why my favourites list isn't the same as my best list.

Personally, I believe we all have our hangups about certain issues where golf and architecture are concerned.  The important thing is that when we are conveying an idea the reader has this as background knowledge to frame the comments.   

Ciao

[Michael Dugger]

I love ya John but Ocean Forest looks like dookie too!

[Patrick_Mucci]

John Kirk,

I agree with the four biases you listed.

Years ago I suggested to Brad Klein that Golfweek introduce a "filter" to eliminate or reduce regional and local biases (repeat play) in their ranking system.

It's difficult to do because most raters will rate courses nearest their home, or nearest a business or vacation destination.

It's difficult to insert a "filter" into the process.

As an Oregonian, you'll probably object to the following.

When I played Pumpkin Ridge I thought it was a nice golf course, but, I didn't think it had sufficient architectural merit to place it in the top 100.

My immediate thought was:  How many MET area courses would I deem to have more sufficient architectural merit, and how many of those courses are currently ranked below Pumpkin Ridge ?

Like the representational balance provided by the U.S. Senate, should states gain representation in the top 100 irrespective of their comparitive architectural merits ?

[Tom Huckaby]

Patrick me lad:

That's all well and good, but John made it quite clear that this is about one's personal evaluations, not how such are done for magazines.  So methinks you are barking up the wrong tree.

That is unless you want to battle the magazine rating again... perhaps the one issue that's been discussed in here more than the origins of Merion.
 

[Eric_Terhorst]

John,

That’s a meaty initial post, very thought-provoking, there may be 6 or 8 threads in there!

I agree that we all bring “biases” to the party when evaluating golf courses, but don’t the best “evaluators” put those aside to measure the merits of the golf holes individually, and the course overall?  I hope so.

Comments:  Following on Mike Sweeney’s comment on his positive bias for sand-based courses (which I share), I have to say that for me target golf is a special category, which can come in parkland and links versions, but more often in desert, or southern resort forms.  I tire easily of courses with lots of water hazards, forced carries off the tee and do-or-die approaches—even though I like visiting (not to play every day) the Pete Dye versions of these that I’ve played.  The wanna-be-Dyes are really aggravating—not worth more than one visit.  If a course I’ve played falls into the “target golf” category and doesn’t have other redeeming qualities, I won’t often go back or recommend it.

I’m surprised your suggestion that eastern parkland courses are all the same hasn’t generated a response from the Philly crowd.  I haven’t played any golf in the northeast, but there is a version of “parkland” that I enjoy—in which trees have been mostly kept in or removed to the background,  like San Francisco GC, French Lick, Lawsonia, to me are quite distinguishable and require more thoughtful design elements than boring tree-lined affairs.  And exceptional topography, as at Ross’ French Lick and Inverness, can distinguish parkland courses. 

I think at least the bunkering styles of Tillinghast, Ross, and Flynn separate them, but I also would be interested to hear the experts chime in on their differing design philosophies.

 I disagree that ‘most golfers’ on repeat exposure would enjoy blind holes, undulating greens, uneven lies, and f&f conditions.  Most golfers don’t appreciate a puzzle like you seem to. 

[Patrick_Mucci]

Patrick me lad:

That's all well and good, but John made it quite clear that this is about one's personal evaluations, not how such are done for magazines.  So methinks you are barking up the wrong tree.

Methinks that you didn't read the first sentence of my post with any degree of comprehension.

The magazine ratings are merely a compilation of ratings/assessments made by individuals.
The magazines colletivize the individual ratings, which I felt were tainted by some or all of the biases John listed.

My post wasn't about magazine ratings, it was about the biases held by individuals, some of whom submit their assessments to magazines.

That is unless you want to battle the magazine rating again... perhaps the one issue that's been discussed in here more than the origins of Merion.
 

My reference to my communication to Brad Klein/Golfweek wasn't about the magazine rankings, it was about the bias inherent in each individual's assessment and response relative to golf courses, and how to OVERCOME those biases when soliciting opinions relative to assessing golf courses.

[Chip Gaskins]

is there a way not to be biased about everything we see, evaluate and decide in life?  seriously, everyone arrives at the table with a bias of some sorts.  good, bad, educated, jaded,etc...but a bias.

[Ian_L]

What about reputation bias?  I know that in the rare occasions when I get to play at a highly acclaimed golf course, I start looking for the great aspects of each hole.  At a few local courses in my area, on the other hand, I focus more on the blandness and lack of creativity required.

[John Moore II]

What about reputation bias?  I know that in the rare occasions when I get to play at a highly acclaimed golf course, I start looking for the great aspects of each hole.  At a few local courses in my area, on the other hand, I focus more on the blandness and lack of creativity required.

Ian--I think reputation and even course designers have a somewhat significant impact on how some individuals rate a course. Like many from this site view Doak and C&C courses as automatically good and Fazio and Rees Jones courses as automatically bad. Reputation of stature plays into the raking and evaluation for sure. I recall someone saying on here that they felt that felt that Pinehurst #2 may not be ranked nearly as high as it is were it not for a well established legacy.

[John Kirk]

Patrick,

Your comments about Pumpkin Ridge do not offend me at all.  I agree.  Pumpkin Ridge does not present a comprehensive test of skill.  Long, steep uphill and downhill putts are rare, and uneven lies in the fairway are almost non-existent.  On the other hand, ambience, walkability, and course flow are excellent.  It's a Doak 6, and a marginal top 100 candidate.

I'm sure there are many (dozens?) eastern parkland courses which offer a more comprehensive test, but I'm not a fan of the homogenous look.  Pumpkin is carved out of an existing mature deciduous and evergreen forest, with generally wider corridors than usually found in Golden Age courses, and is a more attractive park.

There are several noteworthy examples of GCA members touting hometown courses, but after an initial response, I'm inclined to believe the correlation is weak.  The Repeat Play bias is much stronger.  The Remoteness Bias, as suggested by Tom Huckaby, makes sense to me.  It feels special to travel so far from home.  Perhaps there's a psychological need to justify the effort.  But if Sand Hills and Ballyneal were down the street, I'd still try and belong to both!

I'd call Mike Sweeney's bias for sand-based courses a simple fact that they are better.  I'll have to try and add Ballyneal to your list of favorites someday.

Similarly, Carl Nichols' mention of conditioning bias is not really a bias.  Good conditioning yields better golf.  What constitutes good conditioning is a matter of debate.  A second potential bias, Amenities Bias, where the club rolls out the red carpet, is a tough one.  I would propose that some course evaluators are swayed by the red carpet treatmanet, but there is no clearcut evidence to that effect.  Furthermore, around here that sort is strongly discouraged.

Peter Pallotta brings up the concept of experience.  My take on Experience Bias would be different.  And just for you, I'll break out the popular music analogy.  The older I get, the harder it is to impress me with a new song, because I've heard so many good ones in the past.  The same applies to movies and golf courses.  I've seen some great stuff, so it's harder to impress me when I get older.  But when I do find something new and special, it's a gift.

"So as one sees more variety, does one become more "biased" because of likes and dislikes or less biased?"   Mike Policano

I am more biased, because of more pleasant past experiences.

George Pazin mentions Player Bias.  Some courses just don't set up well for certain types of players.  I'm not a real accurate driver, and narrow, tree-lined parkland courses don't set up well for me.  And you know what?  I don't like them all that much, and tend to rate them accordingly.  Am I showing personal bias against them?  Sure.  If you think hitting laser straight drives or punching out from the trees, and flopping sand wedges onto greens is great golf, bully for you.  I don't.  It can be great, but I like wide open, short grass, recover from anywhere golf better.  If George doesn't like a course because he can't play it, then there's something wrong with the course, not George.

"What I meant was something more along the lines of the exchange between Rich Goodale and Tom Doak on Rich's recent "short par 4 made better when lengthened" thread. I think it's almost impossible for 99.9999999% of golfers to look past how he or she interfaces with a course, unless he simply walks it and studies other players."     George Pazin

I agree completely.  My goal is to play golf without prejudice.  I hit golf shots.  I don't care about ratings, or history, or old dead guy architects.  None of that shit matters.  I evaluate a course based on my experience hitting shots and watching my playing partners hit shots, and the enjoyment we derive from that activity.  I don't think there's a bias, just our experience.

[John Kirk]

Good catch, Ian.  The Reputation Bias can go both ways.  Personally, if I play a very highly rated design, I am more likely to find that my high expectations were not met.  I wouldn't say that's true of everyone.

JKM, my experience on the site, as I said before, is that most people here are actually much more open-minded than given credit for.  I perceive the overall analysis of courses here to be fair, and if courses by Doak and Coore/Crenshaw are best received here, it's because they are the best.  Although it sounds like there's a Architect Bias, I believe everybody goes into new territory with an open and positive mind.

[John Kirk]

One more bias, a subtopic bias under Architectural Pedigree, is the Ralph Livingston "If It Isn't 75 Years Old, It's Crap!" Bias.

I'm going to watch a movie.  Thanks for the discussion.

[Tom Huckaby]

Patrick: if you say so.  That sure seemed to be all about magazine ratings to me.  But if you say it's not, then it's not.

John (when you return from your movie):  one thing about this "player bias" that George emphasizes and I say is easy to put aside.  I do agree that there are horses for courses, and that it's easy to get caught up in that.  I never said otherwise.  But I do think it is pretty easy to get beyond that - all it takes is a little thought.  And if George can't or doesn't want to play a course it's not NECESSARILY the course's fault - it might be, in fact it likely is - but it also might be George.  For example George doesn't thinklk much of Black Mesa because he doesn't like desert golf in general, and found that course to be too severe specifically.  Many others find Black Mesa great.  So is it the course's fault that George doesn't like it?  I sure don't think so....  I also have a friend who came back from Ballyneal poo-poohing the course because it had no trees, and he had nothing to aim at (his words) and there was way too much luck involved in those huge roley-poley fairways (also pretty much his words).  Is that the fault of Ballyneal?  Again I sure don't think so... But maybe I am reading way too much into YOUR words.   

I also take a little issue with:

I agree completely.  My goal is to play golf without prejudice.  I hit golf shots.  I don't care about ratings, or history, or old dead guy architects.  None of that shit matters.  I evaluate a course based on my experience hitting shots and wtaching my playing poartenrs hit shots, and the enjoyment we derive from that activity.  I don't think there's a bias, just our experience.

I would phrase that as such:

"My goal is to have fun playing golf.  I hit golf shots.  I don't care much about ratings, but I sure do notice when a course has hosted great things before my arrival - I feel the ghosts around me and it can be fun to walk in the footsteps of the greats or try to recreate their shots.  I don't care much about dead guy architects, as I really don't care who designed a course, but if a course I am playing was designed by the same guy who designed some of the world's greats, that can be pretty cool to me and can make it more fun.  I evaluate a course based on all I experience at a course, which primarily includes hitting the shots and watching my playing partners hit shots, but can also include other things which may add to the enjoyment of the activity that is golf, or detract therefrom.  I don't think there's a bias, just our experience."

So you see, I do believe some of that "shit" matters.  And from the little I know of you, I'd be very surprised if you don't agree with my re-phrasing.  Just remember, I am not over-emphasizing the "shit", just acknowledging that it can matter.

Disagree?

TH

[John Mayhugh]

Good catch, Ian.  The Reputation Bias can go both ways.  Personally, if I play a very highly rated design, I am more likely to find that my high expectations were not met.  I wouldn't say that's true of everyone.

A very interesting topic. 

I agree that the reputation bias can work both ways.  It's much like the JD Power quality ratings for cars.  Some luxury car owners overlook problems because they think their expensive car must be nearly perfect (because it's expensive), while others are the opposite.  They think that for the amount of money they paid for the vehicle, everything has to be just what they like and nit-pick the tiniest details.  I don't really value either of these owners' opinions.

[John Kirk]

Tom,

Let me try to address your two main points.  Regarding your first paragraph, I'm not sure what you mean by the last sentence:

"Is that the fault of Ballyneal?  Again I sure don't think so... But maybe I am reading way too much into YOUR words."

George doesn't like a course, because George doesn't think the course is very playable for him.  So he evaluates the course negatively.  I don't see any bias there.  He simply didn't like it.  However, if he played it multiple times, he is likely to enjoy it more, if it is worthy of higher regard.  But he evaluates the course as the player he is.

Your friend didn't like Ballyneal, which is too bad.  Perhaps if he played it a few more times, or if he was more familiar with dunes type golf, he'd like it more.  Familiarity Bias and Repeat Play Bias may be factors in his evaluation.  I know you'll be playing Ballyneal with your old Santa Clara buddy soon (can't remember his name), so I hope you enjoy it more than your friend did.

Is this the same Golf Digest rater that got a lot of attention for giving Ballyneal an aesthetics rating of 1 because it had no trees?  It's irritating to hear someone say there's nothing to aim at.  You can aim at anything.  I almost never aim at trees anyway.  Remind him that there's no luck in golf, only a range of shots and results.  At Ballyneal, your results may vary.

Regarding the second part, how can you take issue with my own sentiments about golf?  I am interested in the shots presented, the way it looks and the land on which it is built.  Inbetween shots I like to observe the plants and animals during those rare moments I'm not talking to someone.  Those are my feelings.  I really do not care if Ben Hogan hit the 1-iron from here, or if Robert Hunter walked the land there.  It's of no interest to me.  By saying "just our experience" at the end, I may have confused the issue with bad grammar, because I intended those sentiments to be mine.  The concept of Reputation Bias or Architectural Pedigree covers these historical events that most golfers appreciate.

Please accept this response as passionate but not aggressive in any way.

 

[Lester George]

John,

Well thought and well delivered stuff.  Very interesting points. 

Lester

[Tom Huckaby]

Tom,

Let me try to address your two main points.  Regarding your first paragraph, I'm not sure what you mean by the last sentence:

"Is that the fault of Ballyneal?  Again I sure don't think so... But maybe I am reading way too much into YOUR words."

George doesn't like a course, because George doesn't think the course is very playable for him.  So he evaluates the course negatively.  I don't see any bias there.  He simply didn't like it.  However, if he played it multiple times, he is likely to enjoy it more, if it is worthy of higher regard.  But he evaluates the course as the player he is.

Your friend didn't like Ballyneal, which is too bad.  Perhaps if he played it a few more times, or if he was more familiar with dunes type golf, he'd like it more.  Familiarity Bias and Repeat Play Bias may be factors in his evaluation.  I know you'll be playing Ballyneal with your old Santa Clara buddy soon (can't remember his name), so I hope you enjoy it more than your friend did.

Is this the same Golf Digest rater that got a lot of attention for giving Ballyneal an aesthetics rating of 1 because it had no trees?  It's irritating to hear someone say there's nothing to aim at.  You can aim at anything.  I almost never aim at trees anyway.  Remind him that there's no luck in golf, only a range of shots and results.  At Ballyneal, your results may vary.

Regarding the second part, how can you take issue with my own sentiments about golf?  I am interested in the shots presented, the way it looks and the land on which it is built.  Inbetween shots I like to observe the plants and animals during those rare moments I'm not talking to someone.  Those are my feelings.  I really do not care if Ben Hogan hit the 1-iron from here, or if Robert Hunter walked the land there.  It's of no interest to me.  By saying "just our experience" at the end, I may have confused the issue with bad grammar, because I intended those sentiments to be mine.  The concept of Reputation Bias or Architectural Pedigree covers these historical events that most golfers appreciate.

Please accept this response as passionate but not aggressive in any way.

 

John:

1.  Regarding George, you said if George doesn't like a course, the fault is the course's, not George's - plain and simple.  As you explained it here, that makes better sense.  I was just trying to move beyond one's own personal preferences and think about evaluating a course more objectively.  Do you believe that can't be done?  And George most definitely did have bias as pertains to Black Mesa - he made it very clear he doesn't like desert golf in general.

2.  The friend who made these statements about Ballyneal played it several times, and he is quite far from being a GD Rater (though his sentiments were similar to that famous early viewer).  My point was just that this was indeed his view, as clueless and odd as it seems.  So does one take it then that Ballyneal is less of a course, because he didn't enjoy it?  If you stick to your "it's the fault of the course, not him" take as set out re George,  you'd have to say the course is indeed lesser.  Obviously that's not true.  Thus I pointed this out just to question that statement.

3.  I said "take issue" with your sentiments about golf because you worded them rather strongly ("none of that shit matters")... and I figured that a fleshing out of them might give us more common ground than difference, particulary since some of that shit does matter to me!    It would seem that we have a lot of common ground and only one minor difference - I do care if ben Hogan hit a 1-iron from a certain spot, or Watson chipped in, or Bobby Jones walked these fairways... it does matter to me and I feel it.  I definitely feel the reputation bias as you define it (as I guess this would cover the historical things I mention) and to a lesser extent I feel the architectural pedigree bias as you define that.  I would say that there is nothing wrong with feeling these things however, as I think they SHOULD matter... so I have a hard time accepting them defined as "biases".  But fair enough, to you they are biases as you do not feel them.  That is just fine.  Just understand that those of us who feel these things - and I have to be far from alone in this - and think that they matter - would not see them as "biases" but rather just another part of what goes into evaluating a golf course.

In any case, I truly believe we have the rest in common, and to me that's great. 

My response also is not intended to be aggressive, and I have never taken yours as such.  This is all in good fun to me.

And oh yes, I shall be getting to Ballyneal soon - can't wait.

TH

[Jason Topp]

Is there a difference between a bias and a preference?

In other words, one could believe that firm fast courses that allow the ground game to be a factor are superior to target courses.  Is that a bias (which sounds like something to be ashamed of) or a principled criteria for what makes a good golf course?

[Tom Huckaby]

Is there a difference between a bias and a preference?

In other words, one could believe that firm fast courses that allow the ground game to be a factor are superior to target courses.  Is that a bias (which sounds like something to be ashamed of) or a principled criteria for what makes a good golf course?

Jason - that's pretty much my question as well, as set out in that last post.  "Bias" has a negative connotation I just have a hard time accepting about some things, as I say.  In fact I'd argue that some of them ought to be principled criteria for what makes up a good golf course - and I love that wording.

One man's bias is another's principled criteria, I guess.  I suppose we can argue that all day.  Great questions.

TH

[Tags:]

Tags: