The Other Way to do a Ranking

[Sean Leary]

Do any of the rankings throw out the highest and lowest x number of grades to discount for so called biases? If someone is anti/pro Fazio, anti/pro trees, anti/pro water features and puts very low/high grades that skewer the ratings, particularly if not many have seen the course?  What is the average number of raters, say on the GD list, that get to see each new course? If it is 100, and you take out the high and low 10, would that make it "better"?

[Tom_Doak]

Dan:  By "definitive" I only meant one list that would be the most accepted and would not require 200-post arguments on Golf Club Atlas.  I used "definitive" to describe the panel, first.  I do think that is possible to achieve although I agree in the end it is all subjective, as I've said above.

Sean:  Jon Cummings and the other statistics majors will tell you to throw out some high and low grades for "statistical integrity".  Again, I think that's b.s.  Bringing every course back to the mean does no good, in a ranking what is important is whether 4 or 5 panelists DO put the course in the top ten or whether 4 or 5 DO think it's vastly overrated.  You only throw out votes when you don't trust the integrity of your own panel.

Huck:  I wish I could put it together, but alas, it is unlikely.  Maybe I am just trying to convince GOLF Magazine to get back to their original system instead of straying toward the others.  Sure you can settle for 15 passionate golfers who really get around ... there are probably that many on the GOLF Magazine committee now, and you don't even have to pay them ... but eventually they start believing that "they are the panel" and start acting out their importance at a club where they are not a member.  I don't miss any of that, but I can sympathize with Ron and Brad.  Worse yet, most of them are a similar type of player, so the viewpoint gets narrower instead of wider.  The problem with all the committees now is that there aren't enough great players on any of them.

The panelist class might be a tiny percentage but they have a large impact ... I would guess the pro at Crystal Downs fields 5-10 requests a week from panelists to play the course, at least one of whom goes about it the wrong way.  A lot of top courses have come to resent the whole exercise and wish it would go away.

[Tom Huckaby]

Huck:  I wish I could put it together, but alas, it is unlikely.  Maybe I am just trying to convince GOLF Magazine to get back to their original system instead of straying toward the others.  Sure you can settle for 15 passionate golfers who really get around ... there are probably that many on the GOLF Magazine committee now, and you don't even have to pay them ... but eventually they start believing that "they are the panel" and start acting out their importance at a club where they are not a member.  I don't miss any of that, but I can sympathize with Ron and Brad.  Worse yet, most of them are a similar type of player, so the viewpoint gets narrower instead of wider.  The problem with all the committees now is that there aren't enough great players on any of them.

The panelist class might be a tiny percentage but they have a large impact ... I would guess the pro at Crystal Downs fields 5-10 requests a week from panelists to play the course, at least one of whom goes about it the wrong way.  A lot of top courses have come to resent the whole exercise and wish it would go away.

Very sage, makes great sense.  Re the panels and best way to do this, well... maybe there is no such thing.  Whatever way one thinks of, there are bound to be weaknesses.  I guess it's just a fact of golf life these days that these things exist... I just continue to have sympathy for you guys for whom it effects your livelihood.  But then again, you have a lot of positives in your profession also.  

Re the effect of the "panelist class", I'd just counter that for every Crystal Downs there are at least 50,000 courses that never get a single call nor feel any effects from this whatsoever.  Of course there should be zero tolerance for bad behavior, and I hope there is... said miscreants being reported.  And maybe I exaggerate that number.. but hopefully you get the point.

TH

[Tom_Doak]

You don't need to feel sorry for me that the rankings business has so much impact on the architecture business -- I've done quite well with it overall.  But the biases against young architects who aren't household names are still there, and even though I'm now on the up side of that equation, I'd like to see them do better.

My only hesitation in showing up at the GOLF DIGEST summit is that there was a tacit suggestion that doing so might boost my profile among panelists, and therefore the rankings of my courses.  I was just there to help them try to make the system better and to answer panelists' questions about architecture, and of course because I like to listen to myself talk.

[Tom Huckaby]

You don't need to feel sorry for me that the rankings business has so much impact on the architecture business -- I've done quite well with it overall.  But the biases against young architects who aren't household names are still there, and even though I'm now on the up side of that equation, I'd like to see them do better.

My only hesitation in showing up at the GOLF DIGEST summit is that there was a tacit suggestion that doing so might boost my profile among panelists, and therefore the rankings of my courses.  I was just there to help them try to make the system better and to answer panelists' questions about architecture, and of course because I like to listen to myself talk.

TD - understood completely.  My "you guys" was meant to be more broad.  Oh yes, you personally have done quite well... but others may have not, perhaps unfairly.  And that sucks.

Understood 100% re your participation at GD Summit - I'm sure one and all appreciated your taking the time.  And hopefully your suggestions will be heeded.

TH