[Rick Shefchik]

Shivas – You write “Explain to me how raters are charged with impartiality?  The whole point of ratings is to be partial!  That's the point.  There is an egalitarian assumption implicit in this statement that everybody is supposed to get the exact same fair, impartial treatment.  If somebody can show that raters are, in fact, charged with impartiality, or that they're fiduciaries or something, then we can go further, but nobody's shown that, so the statement is inapplicable.”

Was that written by the same Shivas who demands unbiased reporting from journalists? Isn’t a rater, at least in part, a kind of journalist? The rater goes to a golf course to discover facts and report back on what he’s found. He sums it up in an opinion, or a grade. The rater is not expected to be impartial AFTER he has rated the course, but – just as you would hope a journalist is unbiased as he weighs his facts before writing – the rater must be as impartial as possible BEFORE he rates the course. Anything else is tainted by the appearance of influence on the rater’s impartiality.

You insist that there is no evidence that this is a problem, but I think the length of this thread alone says otherwise. Until the ratings panels follow Moriarty’s Laws, with the Kelly Addendums (pay the raters, do not allow raters to identify themselves as such at the golf course), all ratings will be suspect, and these debates will continue.  

Huck – How about if all restaurants began giving free meals to restaurant reviewers? Free=free=free? Yes, but how would the restaurant know when it was serving a critic? The critic would have to announce himself/herself to the management. At that point, how much faith could you put in the critic’s review, knowing that the restaurant, in all probability, went out of its way to prepare a meal and offer service of the highest quality possible – a level of effort few, if any restaurants, are capable of producing for each and every paying customer?

What you have, then, is a tainted review. Maybe it doesn’t affect where one restaurant -- or course -- finishes compared to others, but even if a panel playing free golf says a course is a 9, the paying public might not be getting a “9” experience.

[THuckaby2]

Rick:

Your last paragraph says all that need be said.  I don't think any of these rating panels are giving numbers of stars, or anything like that... they are attempting to RANK golf courses, against each other.  So why does it then matter?  You said it yourself:  

"Maybe it doesn’t affect where one restaurant -- or course -- finishes compared to others, but even if a panel playing free golf says a course is a 9, the paying public might not be getting a “9” experience."

The lists are not attempting to rate the experience for anyone - they are attempting to define the best courses in the land (or world, in the case of GM).

And remember also, Moriarty's Laws, while interesting, ONLY work with Kelly's Amendments.  Absent those, they fail. So perhaps the naming ought to be reconsidered?  I like Kelly/Moriarty Dream World Never Gonna Happen But Would be Cool Rating System, myself.

 
(smiley added purposefully to tweak Dan)

TH

[Dan Kelly]

I like Kelly/Moriarty Dream World Never Gonna Happen But Would be Cool Rating System, myself.

If you really think so, I'd encourage you to communicate that fact to Ron Whitten and his superiors.

Once you've done that, you're welcome at choir practice.

[Andy_Lipschultz]

Shivas: Not sure how it all ends, but here’s a very rough idea of how it could work...

--Publication asks course if it would like to be rated. If so, there are a few conditions (coming up) they will abide by.
--Publication assigns 10 raters to a course (Is 10 enough for a rating?).
--The raters have 9 months (if the rankings are yearly) to play the course. They play it anonymously send their green fee receipt to the publication, which upon getting all 10 rater receipts, sends the batch to the course (blacking out the names on the receipts) which reimburses the publication with one check. The publication could reimburse the rater via a service like PayPal, or hard check. It’s more work for the publication but not something an intern or entry-level employee couldn’t handle, particularly, if it can all be done on the computer.

It doesn’t eliminate all potential landmines (the publication assigning the particular raters is another issue), though I think it’s on the right path. The courses couldn’t suck up to raters, provided the raters didn’t tip their hand and the courses are out of the issue altogether, aside from one check to reimburse the publication.

Rick: On this sort of thread a couple months back, I brought up the restaurant analogy. Mimi Sheraton at the New York Times reviewed annonymously; she never allowed anyone to ever photograph her, lest her face become known to those in the restaurant biz.

The consumer assumes he's reading the truth be it in the New York Times, Des Moines Register or GolfWeek. GolfWeek is part of the journalistic fraternity. Raters may not be, but the publication is and thus should set up its system to be as impartial as possible (though I fully understand not everyone has the $$$ resources of the NYT, LAT, WSJ which gives them a higher perception of impartiality).

[THuckaby2]

Done.  Only it wouldn't be Whitten this should go to.

But shouldn't I also communicate such to Mr. Klein at Golfweek and whomever runs Golf Magazine's panel?

I have as much weight/say in all three of them, meaning, none.

TH

[THuckaby2]

Andy:

Same question to you that I gave to Rick.  Given that no magazine rating system is trying to measure the worth of the experience / quality of the course in anything but comparisons to other courses, why does anonymity continue to matter, so long as they are all playing by the same rules (ie all rounds are comped)?

I buy this re restaurants, because they are giving number of stars or whatever, and thus it does matter to see the restaurant as is.  But since we are just comparing courses to other courses and absolutely NOT trying to give stars or evaluate the course in a vaccum, why again does this matter?

BTW guys, MANY ratings as is are done anonymously....

TH

[Andy_Lipschultz]

Tom:

Announcing one as a rater to a course, can only lead to problems, however slight they may be. It's cleaner and the perception with the public (which is everything) is better. Again, it puts the publication in line with more of the preeminent media outlets (including Consumer Reports) which can only be good for the publication in the longrun.

[Dan Kelly]

Tell me, raters, please:

What is your PRECISE assignment from the magazine that "employs" you?

Is it to evaluate the architecture, solely -- without regard to conditioning or ANYTHING ELSE?

Or is it to evaluate that course's architecture -- and the experience of playing there -- the way a restaurant critic evaluates not just the food and the drink, but the experience of consuming that food and drink?

--

Tom (and Shivas) --

If you are telling us that you are treated exactly the same whether the course knows you're a rater or doesn't ... well, you can't be telling us that, because no one here would believe it.

--

Andy --

Good thinking. Keep at it.

I've brought up the restaurant analogy here, also. It's NOT just Mimi Sheraton. I work at a considerably humbler daily newspaper, without a tenth of the resources of the Times ... and you will NEVER, ever, anywhere see a picture of our restaurant critic (who, by the way, makes reservations under an assumed name and pays with a credit card that does not reveal her identity).

A point worth considering: She never does a review based on a single visit.

[THuckaby2]

Andy/Dan:

I can only speak about the panel I'm on.  We have many criteria we are asked to evaluate, some of which go beyond what people here would call pure "architecture."  But please tell me, why does this matter?  Because sometimes I get to see the course in a better light than the average joe?  Please.  The average joe doesn't PLAY these courses period, do to expense or lack of access.  And the types of courses these comps occur at also aren't the kind that tend to care to gussy themselves up for one freakin' rater visit... Maybe some are, but I sure have yet to experience that!

And again, I still don't get why it matters if I am given preferential treatment on those times I have announced my presence as a rater.  Again, we are comparing courses to courses, not evaluating the courses in a vaccuum.  If they all do this, then what difference does this make?

So we are seeing some the courses at their absolute best. some not.  I guess that's your point.  But just why is this  bad?  All of us are golfers experienced enough to know that courses are great sometimes, not so great sometimes.  Are you also telling me that when we see a course that's been recently aerated - and yes, this does happen - we are too stupid to look beyond that?

Separately to Andy:  I get your point re perceptions of impropriety.  But man, does the end justify the means here?  Think what would have to happen to effect all this... I just don't see it as having any practicality, so it's tough for me to give much credence to.

TH

[Scott_Burroughs]

"Announcing one as a rater to a course, can only lead to problems, however slight they may be"

"It ends when the course you are playing doesn't know you're a rater."

Dan and Andy,

How do you suppose the raters do this at private courses?  Personal contacts will only get a small percentage of private courses seen anonymously.

[Dan Kelly]

Uncle.

Someone with unlimited energy: Carry on.

---------

Scott -- I don't know how to get adequate anonymous access to private clubs.

How did they do it at Friar's Head?

[W.H. Cosgrove]

The raters are one thing, but I would like to hear from courses that accept raters as to what they expect.  

I had the opportunity to have a frank conversation with a pro at a local course that has held a large (major) golf tournament or two.  He asked what what I expected, I gave him the party line that I wasn't to ask for anything except access.  

When asked how important the rating were to the course, he said that guest fees  and paraphenalia purchases from travellers were very important to their cash flow.  He felt that rankings drove a good portion of that and that comped rounds were a small investment for the return.  

This is a symbiotic relationship.  Probably a part of the reason that some of the interchange becomes so heated.  The system isn't perfect but it does work.  It does depend on the personal integrity of those involved.

[THuckaby2]

Dan:

That's funny, we must be a lot alike, because I too was gonna give this one more post and then say "Uncle" myself.  I too am getting pretty worn out by all this.

And it's also funny, I just got through reading Rees Jones' acceptance speech re the Old Tom Morris award, and at the end, he says he is paraphrasing an old saying, as follows:

"Have the patience to accept the things you cannot change, the courage to change the things you can, and the wisdom to know the difference."

I gather you'd say we need courage.  I personally think all that's required here is patience.  But perhaps we both could do with a dose of wisdom.

BTW, Scott's issue is a very real one, one of many which make me call this a Dream. What makes you think the raters who saw FH did so anonymously?  I gather some did, some didn't... several who reported here went as the guest of Mr. Bakst himself...

So what should one do when the practicalities of an idea make it impossible to implement?  Perhaps have patience to accept things he can't change?

TH

[Scott_Burroughs]

Scott -- I don't know how to get adequate anonymous access to private clubs.

How did they do it at Friar's Head?

Dan,

That's through personal contacts and the raters made it a point to get there due to the buzz surrounding it.  That's also just one club.  Many of the top courses would get seen jsut because they're there (and collectors abound).  But there are probably what, 7000 private clubs?  Maybe 500-1000 that could be considered worthy of contention for the lists?   Nowhere near all of them would get seen anonymously by nearly enough people.

[Rick Shefchik]

Shivas and Huck --

I agree that contributing to a compiled list of golf course rankings is not the same as reviewing a restaurant. Different goals, different criteria. But sweeping aside the preferential treatment issue isn't cutting it for me. I'll even adopt the "Tom Huckaby It Ain't Never Gonna Happen" position and say, there will never be universal free access for raters, just as -- for now -- the magazine raters aren't paying for every round. It's somewhere in between -- and that's why when one course gives you preferential treatment, the courses that do not can be perceived to be at a disadvantage. Your magazines could take care of that problem overnight by reimbursing you for your rounds.

And Shivas, you say no one has addressed your issue that a problem hasn't come up yet, but as I said, this very thread proves there is a problem. A half-dozen contributers to this site have expressed reservations or outright distrust for the current systems.  The problem may be merely one of perception, but in journalism, a perception of conflict can be as damaging as the real thing.

You didn't address my issue -- why do you demand less integrity from golf magazines than you do from newspapers and TV news?

[THuckaby2]

Rick:

I'm gonna join the wise Mr. Kelly and say Uncle.  I'm sure shivas has a better answer than I could give anyway.

All I will say is that while that could be done, it has so little chance of actually happening, just because it would cost each magazine SO much money, that it remains sort of silly to me to even talk about.  Then one adds on the very real issue that the ends don't justify the means, to me anyway, and well... Uncle.

But yes, dare to dream.  Good on ya, mate.  So how are you gonna make this happen?  How also are you gonna solve the very real impracticalities of anonymity, as Scott mentioned?

TH

[Andy_Lipschultz]

Scott: Excellent point. Obviously, my scenario is not workable for the private clubs. Quite frankly, I was only approaching it from a journalistic POV; which deals with; how the consumer should best part with their hard-earned $, and thus drew comparisons to restaurants, et al. And to that end, the consumer deserves the best possible system to deliver the impartiality that is implied when something is printed in a publication.

[Joe Hancock]

Shiv,

Is there really 17000 courses that raters rate?

Joe

[Dan Kelly]

... the ends don't justify the means . . .

Exactly what we skeptics have been saying all along.  

[ForkaB]

Just to give a bit of real world perspective, Michelin rates the 18,000 restaurants in France with 5 (count 'em FIVE) full-time inspectors.  Given that the quality of golf courses changes less on a yearly basis than do restuarants, I think that only 3 would be needed to accurately rate all of the tracks in the USofA.  I nominate Dan Kelly, JakaB and GyroGolf.

Any seconders?  We can pay them out of the burgeoning BBGE fund, if Dick Daley gives back his copy of Geoff's book..........

[Joe Hancock]

Rich,

Since you're advocating pay, I'll nominate myself. My pay should be 7 figures, since the livelihoods of so many will hinge on my (and two others') opinion.

 ,

Joe

[ForkaB]

I'll second you, Joe.  AS for me, I'm going after the restaurant gig.  Far more pleasant to die with a coquette on your arm and a bit of truffled foie gras on your lips than by falling over and breaking your neck trying to inspect some asshole bunker........

[Joe Hancock]

Rich,

A-hole bunkers really doesn't sound so pleasant after all. Maybe if I can join you for the food gig, I can learn the nuances of eatting while you have a knitted blanket over your arm and some fake grass on your lips.

Joe

[DMoriarty]

Dan,  the longer this goes on, the stronger my belief that comps might be dangerous.  These free rounds are obviously extremely important to these guys and nothing anyone says is going to convince them to give them up.
________________________
________________________

Shivas.  Your take on impartiality is astounding.  Judges ultimately draw conclusions and issue their Opinions, so I impartiality, so there must be no need for them to try to be impartial.  Might as well try to bribe them with gifts of value, since there is no demand for impartiality.   They probably wont be swayed, but hey, what can it hurt?  

I also liked your desciption of how far courses might go to impress the raters.  I find it interesting that you (and Gyrogolf) diverge from your usual rational economic model when it comes to freebies.   Why on earth might courses go that far to impress raters?   Surely not out of any sort of altruism on their part.

To address your points.

1.  You are absolutely wrong that one doesnt deal with conflicts until there is a problem.  Perhaps one reason I see a problem here is that I have experience ridding businesses of conflicts and payola, and the ex-employees can certainly attest that it made absolutely no difference whether we could prove that the conflict influenced their work.  Some of them can so attest from prison.  

2.  You are also wrong to discount the restaurant analogy.  There are thousands of restaraunts in large cities.  I'll bet Los Angeles has more sushi restaurants than golf courses by a factor of ten.  There are thousands of restaraunts in large cities.  I'll bet Los Angeles has more good sushi restaurants than golf courses by a factor of ten.  Surely in such an environment the critic can remain somewhat anonomous from the ever changing waiting/acting core around town.  

[THuckaby2]

Dan,  the longer this goes on, the stronger my belief that comps might be dangerous.  These free rounds are obviously extremely important to these guys and nothing anyone says is going to convince them to give them up.

Back from uncledom, can't resist....

David, I've said several times I am more than willing to give them up.  Just give me a good reason regarding the very real issues that I've listed several times - that is, absent them this becomes a game for the rich, or alternatively, the expensive courses never get seen. - either way seems to me to make this prohibitive, that is, the new system is worse than the old.  What Dan K. suggested - that the magazines pay the freight - sounds great to me and would seem to solve all issues.  I just don't see any chance of that actually happening.  So do you think the end here - full impartiality - justifies the means - ridding the system of all but the rich - of making for such a system?  If so, then fine, reasonable minds will disagree, again.  Just please do clarify this, so I can finally say UNCLE and leave this in peace.

Thanks!

TH

ps - as for the rest, you two lawyers can debate and I shall gladly stay the hell out of it!

[Tags:]

Tags:

[Tags:]

Tags:

Call to undefined function theme_linktree()