The Other Way to do a Ranking

[JESII]

i think the point is that a diverse group, however small, should provide the wide ranging insight this type of thing needs

[TEPaul]

Tom P:
You are getting off topic, perhaps you should start multiple different threads.  But I'd probably better answer some of those off the board."

TomD:

Good and savy answer, I understand.

[Thomas_Brown]

Talk about photographers, I was very impressed w/ Dick Durrance's passion for the game and golf landscapes when I met him at Bandon.  I'm certain he sees different things than Tom Doak or Bill Coore on the links, but I think I prefer the world through his lens.

Any list would need 4 or 5 superindentents, someone like Drew Annan of Forest Highlands.  It's still a bias, but different than the architect's bias.  By now, it would be hard to keep Bill Coore in the superintendent camp.

[TEPaul]

"Bill Clinton or George Bush 41st"

Boohrah, why don't you nominate the Queen of England or one of her effete sons?  

[Tom_Doak]

Geoff Ogilvy is a great guy and very knowledgable but he has not played nearly as many courses yet as Michael Clayton or IBF.  If I've got to pick two active players, is anyone really going to argue Tiger and Ernie?  And no, they haven't played St. Enodoc; if they had they would probably tell you it's a cool course but really short for them.

Tom Brown:  We always struggled at GOLF to find superintendents who had traveled extensively.  Walter Woods was always on my list back then.  Terry Buchen has seen a lot, too.

Tom P:  No one thinks Joe Kirkwood was a genius as an architect, but all those tours he and Hagen made around the world put them in a unique position in those days.  Plus he was the token Aussie.

[Phil McDade]

Just because someone is a great player does it mean they recognize great design?  After all Nicklaus takes it on the chin on this site.  Would tour caliber players see the greatness in a St. Enodoc?  

Faxon has played, and raved about and appreciated, Machrihanish, a not-unsimilar course to St. Enodoc.

I'd suggest David Owen, because the group needs a good writer (beyond Whitten) with a sense of humor to write all of this up.

[Tommy Williamsen]

Tom Doak wrote, "As for the Musgrove Mills of the world, you are probably right ... I still haven't been there myself, but mostly because my sources have said it's very good, but not THAT good.  We aren't trying to rank the top 300 courses here, that is a folly ... once you get much below 75th place they are all of similar quality and there is no point in trying."

I agree that there really is only a whiskers difference between #75 and 300.  I guess that is my point.  If we are trying to identify the top 100 courses then it is important to play the Musgrove Mills.  At one point MM was 47 on GW modern list.

[Ian Andrew]

Think about the combination required:

1. well travelled
2. knowledgeable about architecture
3. representing a geographic region
4. representing a certain profession within golf

It's easier to pick the best 18 holes from 18 architects representing all regions, than to fill out that panel.

Right now this has become like boxing -too many belts panels and too many rankings. If you don't like the results, you just read the next one until you find one you like. Then you come to GCA and champion that over all others.  

[RJ_Daley]

Tom, that is a fine list, IMO.  For the missing lady, I'd look towards Judy Rankin.  

But, Poppy or Bubba? nah.  

If you are looking for stature, try Sir Boab.    Is there anyone in "Der Bingo's" league?  High profile celeb, entertainer, knowledgeable golfer and well travelled?  Tommy Nacc isn't a bad idea either, IMHO.  

If Ramsey, then why not Keiser as well. I suppose you get into that whole sticky business of dissing one developer for another... Keiser, Bakst, Pascucci, O'Neals, etc...

But, definitely not the Dubai Sheik for a developer rep.!!!

[Rick Shefchik]

"Bill Clinton or George Bush 41st"

Boohrah, why don't you nominate the Queen of England or one of her effete sons?  

Tom P. --

I did, way back on the first page. Tom apparently prefers having a U.S. president on the panel rather than a Royal, but I don't think the Duke of Windsor would agree.

[RT]

What about including some well-seasoned/well-travelled professional tour caddies?

I would think some of them have some excellent knowledge of the courses...

[Jonathan Cummings]

Sorry Tom, I disagree.

Pick your 15 experts give them a ballot and have them vote.  I'll average their numbers and generate a list.  Give them the same ballot and ask us to repeat the process a week later, without the benefit of their previous votes and you'll generate a different list.

Applying a numeric system to a subjective process is inherently flawed - subjectivity will spread your numbers on a given course.  You can't (and don't want to) control this.  But with a small team you introduce a second error - sampling error.  This you can control.

If I have 20-30 votes on a given course my statistics have pretty much converged.  My sampling error is low and more votes have little impact on the calculated averages.

The important qualifications in a rater is that he/she has seen a good geographic cross section of courses, seen top 100 courses, and is willing and able to travel to see needed courses.  Give me a team of a few hundred raters like this and I wouldn't care if their names were Jack Nicklaus or Jack of-all-Trades.

Besides, Jack Trade would be much more receptive to a phone call asking he/she to visit a particularly needed course.  (Can you imagine calling Nicklaus up and asking he run out to see Musgrove Mill?)

A list generated from this hypothetic panel would still be argued over.  But the list would at least have statistical integrity.

JC

[Tom_Doak]

Jonathan:

No offense to you, but "statistical integrity" in the field of golf course rankings is utter b.s.

Here's another concept which you'll understand:  GIGO.

[Gene Greco]

   Two other candidates for consideration:

Ben Dewar - our moderator

Jeff Lewis


Both have played all over the world, including Asia.

[TEPaul]

"Geoff Ogilvy is a great guy and very knowledgable but he has not played nearly as many courses yet as Michael Clayton or IBF."

Geoff Ogilvy gets my attention because he's the US Open champion and he appears to have a thoroughly unique outlook on both golf and architecture for a tour pro.

[Mark_Fine]

I would add to what Ian said about what is needed that any candidate must have seen at least 200 of the top courses in the world.  Then see who is left on the list  

Maybe we need to select a panel to select this panel  

[Noel Freeman]

Tom-

I am bias, but I think Russell Talley is perfect for your panel.  Russell has an almost photograpic memory of golf courses, is one of the humblest, honest people I know with no biases and has seen courses all over the world.  As you know, he's one of the best guys in the architecture world who flies under the radar.

[Jonathan Cummings]

Tom - you miss the point.  Forget golf courses.  Apply sampling error to anything.  Even with GIGO - with enough numbers I eliminate one type of error.  You are right that the other still exists but isn't my "list" more accurate with one type of error eliminated?

JC

[Tom_Doak]

Jonathan:

Sorry, but I don't agree with you.  There isn't such a thing as a sampling error here -- there IS individual bias, but if you pick people whom you trust (and perhaps threaten to out them if they vote for their wallets) this is minimized.  Or, are you just talking about if somebody enters an occasional vote wrong?

There is another math term that might come into play, too.  Significant digits.  If you ask people to rate courses on a 1-10 scale, then the average grade each course gets should really be rounded to an 8 or 9 or 10, and the separations of 0.03 between courses should be ignored, shouldn't they?  Most of these rankings should be pared down to a consensus Doak Scale rating, listing the 10's and the 9's and the 8's, whether there are 100 of them or 67 or 114.

[Tom Huckaby]

Tom D:

For a guy who's been out of the rating game for quite a long time, you seem very interested in it now.  Of course it's understandable.  But is this a precursor to a big announcement?

I for one would find that very cool.

TH

[Dan Kelly]

The other person who has always said the purpose of rankings is "to sell magazines" is Ron Whitten, but this weekend I listened to his publisher tell an audience of panelists about the sanctity of their job and the need for GOLF DIGEST to do it better than anybody else.  So, one of the two of them is lying, or maybe both.

I'm certain that I don't understand (and almost certain that I don't want to understand) all of the politics of this ridiculous business of RANKING golf courses ... but I don't see any contradiction between the need to sell magazines and the need to do well the job that sells magazines.

[Tom_Doak]

Huck:

I am just trying to make everybody really think about it, to shame somebody into making a better effort at it.

The shame is that GOLF DIGEST is best positioned ... some of the candidates for the panel are signed to exclusive contracts by GD, and that prevents anyone else from getting together a definitive panel.  GOLFWEEK has Brad Klein under contract, GD has Whitten and Tiger and Ernie and Watson and Nick Price and David Owen and God knows who else.  But Tiger and Ernie and Watson and Price and Whitten don't vote on the GOLF DIGEST rankings -- you do!  

Years ago, when I started running the GOLF Magazine list, I managed to convince some of the GOLF DIGEST staffers to submit a ballot on the side by telling them we wanted our list to be definitive and we considered their opinion important.  Nicklaus told GD that if other architects were voting on the GOLF panel then he needed to be included, too; and a couple of others submitted ballots surreptitiously (the only one I will "out" here was the late Peter Dobereiner).  But GD told Tom Watson he wasn't allowed to vote on our list and then, to be consistent, they had to throw Ben Crenshaw (who was on GOLF Magazine's staff at the time) off the GD panel where he had served for several years, and the whole thing got much more competitive.  And that's one of the reasons nobody's list is definitive.

I just see all these panels getting bigger and bigger because the magazines think it improves their credibility, when all it is really doing is diluting the vote and adding to the "panelist" class that John Kavanaugh rails on about.  I heard several GOLF DIGEST panelists last weekend whine about the exclusive clubs that didn't want to have them, completely oblivious to the clubs' point of view.

[Dan Kelly]

... nobody's list is definitive.

You think a "definitive" list is possible?

You think it's possible to say, *definitively*, whether Pacific Dunes is better than Ballyneal, or Ballyneal is better than Pacific Dunes?

[Mark_Fine]

At one point in time, Golf Digest asked a number of individuals (it actually included some of the names that have been recommended here) to list the Top 10 golf courses.  You would be amazed is some cases what they came up with    

Honestly, I'm not sure what would be gained other than another Top 100 list for everyone to argue over.  For example, guys like Jack Nicklaus haven't even seen another Tom Doak course.  How is someone like Jack going to rank the Top 100?  

[Tom Huckaby]

TD:

Damn, reality rears its ugly head again.  I am naive enough to think that you really had a way to make this super-panel happen.  And I was excited about it, because to me it would have been a hearkening back to the old Golf Magazine days at least a little... and those were great panels indeed... weren't they?

But this is big business now (at least partially YOUR fault, btw   ) so it is what it is and with all these contracts you couldn't get the right people together.  Oh well.

BUT... as I alluded to in the other thread, couldn't you get an equally well-travelled, perhaps more passionate, damn good panel of 15 among the "not so famous"?   None of these would be under any contracts... Of course reality rears it's head there too... coverage would be an issue, funding, how to find them, etc... but that has to be intriguing.

In any case re Golf Digest, why is it not just fair to look at it as ONE way to do this - no better or worse than any of the others?  Of course I find it very cool they continually tweak and try to improve... but in the end, GD does a consensus of many, GM is the view of a few (or at least was - I'm not privy to how Passov will re-do things), GW seems to be the views of a larger few, focusing on slightly diffrent things... Thus three ways to do this, three different results...

BTW, the concerns about the "panelist class" seem to be to me much ado about nothing, as much as I get a kick out of JK's crusade against this. Please... put all three panels together and what do we have, 1100 people?  Now I don't know how many avid golfers there are in this country, but that has to be a TINY percentage of them.  If that's a class, then so are Santa Clara Bronco women's soccer fans.  

And of course of these panelists, there are always going to be bad apples.  Sadly you met a few last weekend... hopefully you met some of the good ones, also.

TH