GOLFWEEK: strictly by the numbers?

[Tom Doak]

For Brad Klein or Jon Cummings:

I am always a bit curious whether the final results we see in GOLFWEEK are the actual, unedited results of your poll.

Last year you had a couple of curious, lesser-known courses in the 98-100 spots [Back Creek in Delaware was one of them].  This year it's Kingsley Club and Barona Creek.  I'm not questioning whether those courses DESERVE to make the list. I'm just wondering if they actually did, or if the editors somehow found room for them at the end, at the expense of some other designer who was already well represented.

People always questioned whether my work for GOLF Magazine was strictly on the up-and-up.  It was published exactly how the numbers came in.  Tell me yours is, too.

[Nick_Ficorelli]

Tom:
I know that I personally spoke with or accompanied enough raters to qualify at Kinglsey Club....I would think Gib did the same at Barona, from the feedback I got.

[Tom Doak]

Nick,

I wasn't doubting whether enough raters got there.  I'm wondering if one of those courses didn't really finish #102, and get bumped up slightly in the final accounting.  That would bother me, even though I know as well as anyone how little actual difference there is between #100 and #102.

[Tom Doak]

But, you raise another interesting point -- should one or two raters accompany others around a course they like?  Sounds as though it would be pretty easy to convince a fellow rater to like something you liked.  Ten independent opinions of a course would carry more weight, wouldn't they?

[A_Clay_Man]

It's funny you mention padding the final result. I always thought it was GD that somehow had a backroom deal going on when it came time to publish.

[Nick_Ficorelli]

Tom:
I see no problem with raters playing together nor have I experienced raters attempting to influence each other.
Playing with anyone could have an influence, ..ideally, every rating experience would be alone with an added walk thru ...
not always practical.

[redanman]

In my experience as a rater for GW, the courses often try to get me to get other raters to "fill out a foursome", especially at the new courses. (One reason I try to play at really off times is to get no flak for going out alone).

I'll admit that I do play with about 3 or 4 other raters who live near me when we can.  Several together can only help each see more as there is really a lot to see on the really good courses.  I try to play more than once if I can and I always try to look outside the box of where my ball, today's pins, what tees I played, how I hit it, blah blah blah expose me to the course on a particular visit.

I've said before, I like to play with my wife who gives me a totally different look at the course than another 5 hcp, a 2 or a 9 playing the same tees as me, or with women in general.

As for playing with the other raters with which I play, we really form our own opinions, I don't think any of us has ever bullied anyone else along to a higher or lower number or a uniform number for a particular course.

The architects themselves with whom I've played or candidly discussed their courses (founder of this thread included) with will tell you off the board, I am not afraid to let my honest opinions be known.  

[Gib_Papazian]

Tom,

Barona made the list not because Tommy and I spread the word, but because it is outstanding.

In truth, Tommy found it first and I actually rated it higher than he did.

[Paul Richards]

Tom:

My favorite way to play a course is not with another rater,
but with someone who actually knows the history of the
course - club historian, superintendent, long-time member,
or whatever.

The more information about why things are the way they are
only helps me understand the course tremendously.

[JohnV]

Having played courses with a number of other raters, I can't think of an instance where we discussed rating numbers while playing.  We did point out features and discuss what we liked or disliked at times.  If there was a time where there might have been more "influence" than others it was when some people really didn't like the course and it might have rubbed off on others in the final numbers.  But, even in those cases, I doubt that any of us would have rated the courses very highly.

As for Barona, I played Barona last spring and ran a Futures Tour event there, so I saw it for an entire week and my opinion only went up over the week as I watched good players work at figuring out some of its nuances.

I hope that Brad doesn't maniplulate the numbers to make certain courses appear on the list and I don't think he does, as it would certainly damage the credibility of the process.

[Patrick_Mucci]

Tom Doak,

I had written some letters to the magazines asking the same basic question from a slightly different angle.

I was interested in how they modified or offset the homer tendency in the rating process, and if there was any regional or national editorial review.

I believe the response I received indicated that it was just the numbers, but there was an interest in having review panels to avoid stacking or homerism.

That was a couple of years ago, so I don't know the present arrangement, and would hope your question is answered.

I have always been more interested in the PROCESS for determining ratings, then the actual results.

[Matt_Ward]

Pat:

The issue you raised is a good one.

Like you, I have always been concerned that homerism is something to watch -- particularly in areas where secondary courses sometimes gain because of superior heavyweight layouts nearby. These secondary courses can often have inflated numbers.

Even though the NY metro area has many fine courses I often believe that plenty of these courses are simply rolling along with the same ratings because "they have always" been rated.

I don't do that when I review a facility and to be totally honest I can be extremely more demanding when I'm in an area of the country where the possibility of homerism can be a legitimate factor to watch and monitor.

Watching numbers on golf course ratings is no less interesting than watching numbers on the ice rink and can be as controversial. But, Pat ask yourself this, how does anyone know how the next Pope is selected until the white smoke puffs out from the top of the building? Your question on process is something that is most crucial because at issue is the credibility of the entire effort not just at GW but at any publication.

[Mike O'Neill]

Tom Doak,

Where exactly does your original question come from? There must be some mistrust of the system. Is there anything in particular that sparked the question? Is this specifically about the last couple of courses or are you concerned that the same manipulation has happened in order to get a course or two into the top ten?

[W.H. Cosgrove]

Tom asks an interesting question.  As a GW rater I am always concerned and aware of any implied or apparent slight of hand.  

However, there are always going to be situations where courses are either rated too high or too low during the process.  There are only 175 raters for Golfweek and it takes time for enough raters to see a course before it settles into an appropriate spot. It is simply a matter of the law of large numbers.

What Brad and GOLFWEEK have done is to make a concerted effort to educate the raters by offering Seminar/outings to locations throughout the nation.  These outings are not simply excuses to play golf.  Much time is spent in the evenings working through issues and utilizing case studies in an attempt to give each of us a basis with which to rank courses.  

Last fall we gathered in Indiana and played French Lick and Victoria National.  At French Lick we discussed possible areas in which to concentrate their renovation efforts.  This process was made more interesting by the French Lick staff as they were quite forthcoming about their particular issues, most of which were financial.  As raters we were allowed to make a real life examination of the issues involved rather than simply making unsubstantiated suggestions.

Tom Fazio was scheduled to join us at Victoria National. Unfortunately due to plane problems, he used the phone to speak and answer questions.  

These two couses could not have been more different.  The raters in attendance and who have attended previous events at Dafuskie Island and Bandon are begininning to work from a common set of data.  Brad and GOLFWEEK should be applauded for the effort.

For those of you who might make the leap to thinking that we are being indoctrinated to the GOLFWEEK school of rating.  Jonathon and I rarely agree on course ratings. Brad and I have butted heads on multiple occasions.  Fortunately after the several thousand ratings are added up and averaged I believe the list becomes more representative every year!

[ian]

In the early 90's, I assisted with Score's Magazine's ratings for Canada. I got to see all the votes. Like all magazines (what have they to gain?) the rankings were true to the votes. I can also tell you that there was a huge bias for the local region. The bias throws off the rankings.

[JakaB]

Tom Doak,

What has motivated you to ask a question you already know the answer to in a public forum.

[David Wigler]

Tom,

I am not going to touch the first question you asked.  Brad is the only one who can answer it.  My gut is that Brad would never do anything that might jeopardize the credibility of his magazine, like make up ratings.  I cannot speak for Kingsley (Although I plan on playing it early this year) but Barona certainly belongs in the top 100 and since neither course advertises with Golfweek, I cannot think of a reason why the magazine would jeopardize credibility to place them over a more deserving course.

I will try to answer your second question.  I have found that playing with other raters has no effect at all on my opinion.  I played Victoria National with three other raters.  We had four very different opinions on the golf course.  It results in a fantastic 18 holes of debating its merits.  It was a treat.  I do not think any of us convinced the others of our position but we all tried our best to educate on what we felt worked or didn't.  I have played a lot of golf with other raters and it has never once occurred to me that I have to agree with them.

I was surprised mostly that you asked the question.  It almost implies that your hidden motivation for archipalooza was that you could get ten raters together and influence them that Pacific Dunes belonged in the top five.  Out of curiosity, to you believe Pacific Dunes is worthy of its high ranking or do you think it got it by having a bunch of raters play it with you?  From what I have heard, Pacific Dunes is absolutely deserving of its place and that fact that you had groups of raters play together and with you had no effect on its placement.  If only one rater was allowed to play at a time and events like archipalooza were automatically disqualified, I doubt Pac Dunes would have had enough votes to qualify in either Golf Magazine or GW.  That would have been a shame, as a lot less people might know to go there.

[jim_lewis]

Tom:

Are you asking a question or making and accuasation?  If it is an accuasation, please say so. If it is a question, I suggest that it would be more appropiately addressed to Brad in private. How would you react if Brad or Ron asked you in public whether you doctored the GOLF ratings?

[Tom Doak]

Actually, Jim, more than one architect and one writer accused me publicly of "influencing" the final GOLF Magazine rankings, even though it wasn't so.  I didn't mind explaining to them that we stuck to the numbers.

I'm not "accusing" anyone of anything.  I asked the question because I know how frustrating it can be to be in charge of a list and to not agree with it.  (I wished in many cases I could have struck things off GOLF Magazine's list, and I know Ron Whitten felt the same about GOLF DIGEST's rankings from time to time.)  I just found it an odd and fortunate coincidence to have those courses which some people consider "under-appreciated" slipping in right at the bottom of the list.  I wanted to hear some reassurance about the sanctity of the process, for the same reason people wanted to hear from me.  I would hope Brad Klein wouldn't take offense at that, any more than I took it from him.

I don't have any qualms or suspicions about how the top of the list comes out; I can't imagine why anyone would be tempted to change those positions around.  Everyone likes to talk about what's #1 and #2, but in that rareified air it doesn't affect their business very much.

[David Wigler]

Tom,

Thanks for explaining.  I feel better about the question.  

I have a question for you.  Do you feel that your presence influenced the Golf list?  I will try to explain.  Obviously you ran the list and decided whether people remained as raters.  This would be a very precarious position to then have to evaluate your work.  I often had doubts about Golf's list.  I never felt you would cheat but I could not imagine how someone working for you could objectively evaluate you.  I suspect that GD is going to have the same problem this year with Architects Club and I am curious how Ron will handle it.

[Richard_Goodale]

Tom D

I'm with you on this one.  While I do not at all dispute that Barona probably deserves a high status (I will confirm this in 5 weeks or so ), I too find it a bit coincidental that it appears at exactly #100, given that it is a fairly unknown track (outside of this DG), and has gotten a lot of ink on this DG, which is hoachin' with GW raters.  If Brad, or whomever, would tell us who was 101, 102....110) and why this year I might be less sceptical.

[Dan Kelly]

(1) Is anyone getting the picture here?

Beyond its being totally subjective (no matter how many variables you measure, and no matter how many decimal places you carry each measure to), this whole business of RANKING courses -- especially public-access courses, it seems to me -- is an invitation to corruption. (Or, if you like, to "fudging the numbers.")

Where (if anywhere) is this invitation most likely to reveal itself? At the bottom of the lists. Naturally.

This isn't the Fortune 500, where (supposedly audited) numbers tell the story.

These lists are the product of thousands of individual transactions -- over which the sponsoring publication has little, if any, control. Am I wrong?

Note well -- and I mean this most sincerely: I accuse no one -- not in the case under discussion; not in any case. And I doubt no one's integrity. I mean to suggest only that imperfect people sometimes make imperfect choices when operating in imperfect systems.

I say: Let the folks at Golfweek and Golf and Golf Digest make a big deal of their lists -- but why should we?

(2) I do think Tom Doak deserves an answer.

[jim_lewis]

Tom:

Thanks for your reply. I too, feel better about your question, although I hope you see how it could be interpreted as an accuasation disguised as a question.

Call me naive but I have never suspected you, Ron or Brad of doctoring the rankings.  I know for a fact that there are some courses on the GOLFWEEK lists that Brad would prefer not be there and I know some that he wishes had made the list.

I also recall a conversation with Ron several years ago when he told me that he was embarrassed that Grand Cypress was on the GD list, but that he was obliged to let the chips fall where they may. Besides, I can't imagine that Ron agrees that PB is the best course in the U.S.

[Rick Shefchik]

The comment in Tom Doak's most recent posting that caught my eye and got my mind working was this:

"Everyone likes to talk about what's #1 and #2, but in that rareified air it doesn't affect their business very much."

Unwritten, but obviously implied, is the idea that scraping into the top 100 at #94 or #97 or #99 would have enormous effect on the business of those fortunate courses. With that much at stake, it's no wonder that questions arrise about the subjective effort to rank golf courses -- which, after all, are much more than golf courses. They are investments, corporations and even regional economic anchors.

Ranking golf courses is a much more sensitive and complex endeavor than ranking baseball players, pianists or surfing beaches. My hat is off to those of you who do it with integrity and utter disregard for influence.

Rick

[Ron_Whitten]

In answer to David:   The Architects Golf Club is NOT eligible for any Golf Digest course rankings. That's how Golf Digest will be handling it. That's my deal with them, and the owners of The Architects knew that going in.  I was retained with that understanding.

I'm sure Brad had plenty of evaluations and great numbers for Kingsley and Barona Creek. I'm more curious about the state by state rankings. Having done those for Golf Digest since 1985, and knowing how hard it is to get a minimum number of panelists to each state candidate course (and Golf Digest has 800!), I'm curious about how many REALLY played some of these state-ranked courses. Raise your hand if you've ever played Chisholm Trail.  Anybody? Anybody?

Hey, I'm a fan of Chisholm Trail, and have promoted it in my  writing, but I was very very surprised to see it listed third in Kansas.  Even among public courses, which is what the GW state rankings are limited to, I've got to believe many other public courses in the state would outpoll it. Fess up, Brad. Did you seed Chisholm Trail onto the Kansas list?