Responsibility and course rankings

[Matthew Mollica]

This discussion shows up the great fault in ranking lists. They are subjective, but many view them as objective.

In other words, many (including golf club committees) view these lists as being accurate accounts of the quality of the golf courses in that particular country. Even though, these same people have probably never seriously considered what ‘quality’ is. They assume the people on the panel know what they are talking about & play all the courses on a regular basis.

I think subjectivity is a great thing, but when people view these lists (& magazines promote these lists) as the definitive ranking, then it can cause problems.

The lists create discussion, but at what cost ?

Andrew's correct, in that rankings are subjective, and influencial. They're embraced by many, and often precipitate committee level decisions resulting in significant change to many courses over the years. Sometimes for the wrong reasons.

In compiling these lists, I'm not aware of a single publication which provides detailed written feedback to the Clubs whose courses are up for consideration. Many produce pages justfying the form of their annual / biannual rankings editions.

Is it the role of magazines to comminicate their views to the Clubs?

I feel it would be a beneficial and responsible thing to do. Some would no doubt feel it's the publications big-noting themselves. Some Clubs would simply dismiss such communications, and rightly so in some instances. Yet I feel such an effort should be undertaken to either aid future course remodelling, or safe-guard some Clubs from blindly and / or erroneously taking the scalpel to their courses, in the name of attracting elevated status in the minds of rank and file golfers.

It may even see the rankings embrace a slightly more objective mode of assessment.

What do you think?

Matthew

[John Kavanaugh]

Matthew,

How is a guy like Paul Thomas qualified to give a club advice? 

[PThomas]

we can't all be as smart as you john

[Brad Klein]

Actually, I find Paul's judgments to be thoughtful and consistent. And respectful of each course. He's also a lot more mindful of the relationship between playability and ecological sustainability than most knowledegable golfers. Does that qualify him as a consultant? Not necessarily. But it does qualify him as a rater.

The collective wisdom of people like him makes for credible ratings. Matthew's suggestion is excellent, and we are working on the ability to extract complete data on every criterion so we can communicate with clubs that ask for it.

[John Kavanaugh]

Actually, I find Paul's judgments to be thoughtful and consistent. And respectful of each course. He's also a lot more mindful of the relationship between playability and ecological sustainability than most knowledegable golfers. Does that qualify him as a consultant? Not necessarily. But it does qualify him as a rater.

Exactly.  I wish he would have told the powers at Butler National not to remove the trees in the wet lands on number 7 because of the fragile ecosystem.  The golf business is in dire enough straits without having raters trade consulting services for free rounds of golf.  As Paul might say..This could be a slippery slope.

[John Kavanaugh]

The collective wisdom of people like him makes for credible ratings. Matthew's suggestion is excellent, and we are working on the ability to extract complete data on every criterion so we can communicate with clubs that ask for it.

I would fully support a system where each rater wrote a paragraph like you do in your raters notebook and those would be available to each club.  In that event I believe the club, no matter what the ranking, would come out the winner.  With contact information of the rater club managers and the like could possibly get more extensive feedback.  It's all a win/win as both clubs and raters improve.

[W.H. Cosgrove]

GD and GW have been undertaking an effort to educate their raters.  GW has 450 raters and requires a minimum number of ratings to give a course a 'final' number.  This attempts to use the laws of large numbers to minimze the effects of one skewed number by a rater with either an agenda or who is having a 'bad day.'.

While the the system isn't perfect it does provide a good deal of valuable input for readers of magazines.  As I see it the biggest issue may well be placing 'amateurs' and enthusiasts into the role of expert and somehow using that information as a reason to drive change.  I believe  both magazines have begun their attempts at continuing education to create a product of value to the magazine and their advertisers, not the courses themselves. 

The programs are always going to have inherent issues, but certainly provide grist for the GCA mill.

[Mike_Clayton]

Matt,

In the case of Golf Digest here in Australia they should tell the club just how many of the 60 odd raters actually saw the course.
My bet is less than 10% saw anything of the work we have done at Peninsula North and South,Lake Karrinyup (in fairness unfinished at the time of publication) Royal Queensland and The Grange West.

And they need to be able to justify how St Michael's rates ahead of Portsea - because if 100 blokes on this board saw both 100 would vote for Portsea.There is nothing subjective about it.

[Jim Thompson]

Having been though the process, I really appreciated the approach of one particular rater.  Although I didn’t agree with him on every item, I believe John Percival should be the model for the methodology and behavior of a real course rater.  I also thought David Wigler did an excellent job but don’t want that to influence the credibility of his conclusions.  The bean counter in me, is often obsessed with process and methodology, and I think those two embraced those items most admirably and to the best of their abilities.  I can't be too objective with regard to Paul 'cause we just had too damn much fun!

Cheers!

JT

[Tom_Doak]

Please, God, no.  The LAST THING IN THE WORLD clubs need is to see a bunch of data from raters.

It is just a beauty contest.  Some of the raters are clueless, but leaking their collective wisdom to member-run clubs would prompt that many more meaningless changes to courses.

The only worse thing would be to have multiple architects write up their evaluations of a course.

[Ray Richard]

I agree with Tom D. I had a course rated once by GD and they sent me a copy of all the rater reports-it was a collection of foolish design interpretations and incorrect conditioning comments. I filed it away and didn't show anybody, for obvious reasons, and the course has had saturation play since opening.

[Melvyn Morrow]

‘Beauty is in the eye of the beholder’

Tom, I believe you are right.

GCA.com proves the point - that there are a variety of tastes and golfing opinions that can, at times, be more frustrating than debating politics.

Rating and ranking a course is as difficult as comparing golfers or designers from different ages. 

[Matt Waterbury]

The only worse thing would be to have multiple architects write up their evaluations of a course.

That's why we come here...

mjw

[Matthew Mollica]

I agree with Tom D. I had a course rated once by GD and they sent me a copy of all the rater reports-it was a collection of foolish design interpretations and incorrect conditioning comments. I filed it away and didn't show anybody, for obvious reasons, and the course has had saturation play since opening.

Ray, Imagine if a course fell in ratings, and a committee found their club on hard times, and received the same documentation you did. Their temptation to alter the course to stimulate greater playing numbers would be strong, but hopefully resisted in the event that rater feedback was deemed not worth a hill of beans. The feedback process would then have merit would it not?

Tom, I'm not talking about an expansive text. It could fit on a business card if need be.

MM

[corey miller]

As a stakeholder in a club that just went through the restoration/renovation process (though it is ongoing) I would most certainly be interested in seeing the  specific thoughts  of  the raters.  In fact, I have a lot of respect for the views of the few I know personally and I do solicit their opinion from time to time. 



 I also know the ratings themselves can be flawed and I  really don't take them too seriously but I do respect the  fact  that any "surge" in the ratings   could serve us  in our effort to  memorialize the changes that Bahto and Hanse have made to our  golf course.


All that said, and this is the reality of the situation, if you had to fight with the membership to get them to listen to Hanse and Bahto who spent much time on the site, why would you want or seek for them to be influenced by someone doing a four hour fly-by? 

These comments in the hands of the wrong sorts of people
that exist in all clubs could be dangerous.

[John Gosselin]

"These comments in the hands of the wrong sorts of people
that exist in all clubs could be dangerous."

Corey, as a superintendent I couldn't agree more.

I personally played with 11 raters in 2008 as host for the club where I work. For some reason some of our members viewed these people as some type of expert in not only architecture, but also in agronomics.

I enjoyed playing with about half of them. I didn't exactly agree with everything they had to say, but they did there homework before coming to play and we had some thoughtful conversation. The other half had no business being raters.

IMHO the credibility in any rating system lies in the quality of the raters.






 

[Adrian_Stiff]

The subjective rating of anything is always going to be controversial, you are always going to get a silly opinion and opinions that are biased or even 'bought'. Older golf courses will always do better because more have played or seen them and the ratings nearer the top are likely to be more solid. Sadly in a commercial world ratings are very important.

[J. David Hart]

John G, your comment from a super's perspective,.. even with the best budget the variables you face daily is a true indicator. To many of the unknowing beleive opinions just like they are fact. In golf there are far to many variables to ever put to much into an opinion. Anyone should take the time in rankings, and ratings to temper their opinions, responsibly, with a disclaimer of sorts. That day , that moment,  may never be duplicated again. Ponder what Doak said, a while back concerning Scotish golf....where
golf is still golf. Is our systems of rankings perhaps a little to one dimensional?

[wsmorrison]

Are you serious?  Letting raters send comments to the clubs is the height of folly.  Higher even than allowing raters special privileges not granted other guests, except courtesies extended to industry professionals. 

Just how is a club that doesn't know Rater A from Rater Z supposed to consider the credentials of the raters and give their opinions based on a single visit any weight whatsoever?  If clubs are going to start making decisions based on the exit polling of raters, they're in for a lot of trouble and merit the consequences of such idiocy.  Industry professionals, determined to be experts after proper due diligence is the only way to make decisions.  Just make sure the historical research is comprehensive 

[John Gosselin]

J. David Hart, you are absolutely right about the one dimensional. The course playability  changes daily and we should embrace that variety. It keeps thing interesting. 

Wayne, your comments are the ideal, but the reality is often different.

[Karl Bernetich]

I guess I sould be taking this discussion in as part of an Architecture pov
But I'm not ...

Should there be more than 1 catagory for the ratings ?

There are some very good courses w.r.t. Golf Course Architecture - Catagory A
There are some very nice CLUBS - Catagory B

After my last "Club" experience, I'm inclined to lean towards the latter for my preference in playing and joining a club.  The current ranking seem to imply high in Cat A is high in Cat B

Combine Catagory A & B for the best-of-the-best.  There are some clubs that are BOTH -- and those are the one that don't seem to be  in membership trouble.

[Adam Clayman]

Andrew's correct, in that rankings are subjective, and influencial. They're embraced by many, and often precipitate committee level decisions resulting in significant change to many courses over the years. Sometimes for the wrong reasons. How is it the magazine's issue if people view the rankings in such a light? Influencing changes to architecture should be each clubs responsibility to get it right the first. Unless of course you have the money to do it all over again.   

In compiling these lists, I'm not aware of a single publication which provides detailed written feedback to the Clubs whose courses are up for consideration. Many produce pages justfying the form of their annual / biannual rankings editions.

Is it the role of magazines to comminicate their views to the Clubs?

After reading John Gosselin's post, I'm struck with the idea of having the golf clubs rate the rater.

[John Kavanaugh]

I like the idea of every rater writing a comprehensive report exactly like Klein does for his publication.  I also like the idea of the club having access to the report along with the raters name and phone number.  This type of hard work would separate the PT's from the 109's.

I do think the good people at Butler National deserve an explanation beyond you should have hired Hanse.

[Lou_Duran]

I would think that more information properly organized might be helpful to the clubs' decision makers.  It may also serve a useful purpose for the folks running the ratings in evaluating its panelists and pointing to areas where additional training might be called for.

For the most part, people know that the ratings are subjective and, therefore, of only limited use for recommending action.  I've yet to read any claims by the magazines which suggest a level of precision or science that is not there.  Some courses themselves may misuse the rankings in their marketing, but that's a different issue. 

[Chris Kane]

Matt, the end result of the AGD ranking is a clear indication that most of them don't have a clue.  Any panel that had St Andrews Beach at #29 and Ranfurlie at #94 is not one I'd want have any influence on decisions at my club. 

I agree with Doak et al that asking raters for feedback is asking for trouble.

If clubs want to improve their ranking, they should seek advice from a golf course architect.