Should raters be rated?

[Mike_Young]

As with anything there are always the whack jobs in any field, hobby or vocation.  And it is often the whack jobs that get the attention.  I have seen complete whack job writers  get credentials for major championships and  whack job raters smooze  their way across country on a golf trip with buddies w/o paying a dime for golf/lunch for the entire group.  
I usually side with JK when it comes to such but in saying that I think it is usually the whack jobs he is describing.  There are plenty of good raters and writers that go about their task in proper form.  I'm not talking of those...but if something like the below was implemented across the board it would be good for golf writing and rating.  And many of the whacker golf writer/raters would quit.

The food critics now have a set of guidelines for all food critics not just one magazine.  Why couldn't the same be done for raters and writers and all be vetted.  The below are pretty good...

http://www.afjonline.com/FoodCriticsGuidelines.cfm

http://www.lamag.com/lafood/digestblog/2013/05/14/new-guidelines-for-food-critics-acknowledge-bloggers-the-internet

[Patrick_Mucci]

Mike,

What a great topic.

Unequivically...........YES

And, if that can't be done, then, prior to being appointed, they should have to submit their curriculum vitae.

[BCowan]

Mike  

   Another great post.  It makes too much sense.  If I am not mistaken raters don't walk the course.  Most just basically play it.  I don't see how one can get enough time to scope out nuances while playing.  The restaurant idea is great.  Funny I had a lady play in my walking event last year that was a food rater, I wonder if she paid to try the famous ann arbor deli?  I hope you start this for golf arch

[Peter Pallotta]

deleted - apologies if i offended anyone, a joke amongst friends that was taken badly.

Peter

[Mac Plumart]

Mike...Frankly, I think that is spot on.  If we are going to set forth on a mission to put forth the most unbiased opinion regarding the quality of a golf course, similar guidelines to the ones you post should be used.

Question...do these Restaurant Critics cover a small regional area or are they Nation/World wide? 


BCowan...Why do you say raters don't walk the course?  I, personally, walk many of the courses I rate.  I know many other raters do as well.

[BCowan]

Mac

   I meant to ask a question if they do?  Sorry about saying most don't.  I only caddied for two raters and they didn't as I recall and it was 14 years ago.  I hope they do walk the course when they rate.  I did not mean a blanket statement. 

[Pete Lavallee]

I, personally, walk many of the courses I rate.  I know many other raters do as well.

Just curious, what percentage of courses you have rated did you walk only?

[Mac Plumart]

Gotchya.

I actually think that gets into an interesting point on rating.  What is the purpose of the rating process?

Is it to find the best course for a specific subset of golfers?  Or is it to find the most universally appealing course?

Most universally appealing course...I think you need a cross section of golfers.  Walkers/Non-Walkers.  Scratch players/Hacks.  Men/Women.  Great Drivers/Great putters.  Then take all their opinions and sort the data.

Best Course to a Specific Subset of Golfers...obviously, this kind of approach appeals to a much more narrow audience.  For instance, Best Course for a Low-Handicapper.  Best Course for Walking Golfers Only.  Best Course for the mid-to-high handicappers (perhaps labeled the Most Playable Courses).

[Adam_Messix]

Mike

Who says that raters are not evaluated?

[Patrick_Mucci]

Mike

Who says that raters are not evaluated?

Adam,

You're implying that they are, so please tell us how "Golf Digest" and "Golf Magazine" evaluates their raters ?

How much annual turnover is produced by those alleged evaluations ?

[Tim Martin]

Mike-I agree is a good idea but unfortunately anonymity and access in the case of private courses are not mutually exclusive. In order to rate the golf course you have to arrange access at which point your cover is blown. I would think that this facet of any proposed guidelines would be most beneficial to the process and yet unattainable at the same time.

[Mike_Young]

 What is the purpose of the rating process?

Mac,
That is the million dollar question for the industry.  I have my thoughts for the different publications.  BUT if guidelines such as the food critics outlined were followed the system would improve for all IMHO.

Adam,
I'm assuming they are not evaluated by anyone other than the particular magazine for which they work,  The food critic outline was generic for critics working for all magazines.  Also, Adam, for those of you raters that are not part of the segment I am discussing, I think it is difficult for you guys to imagine what is seen out there sometimes.  But as an example, I have had friends who were raters come to a club where I am a member and tell the pro they were a rater expecting not to pay and yet having no reason to rate the course or write it up.  They came back several times that year and did the same.  I never let them know I paid for each time this happened.  I'm not saying the guy is a bad rater but under the food critic guidelines this would never happen.  

[John Kavanaugh]

Mike-I agree is a good idea but unfortunately anonymity and access in the case of private courses are not mutually exclusive. In order to rate the golf course you have to arrange access at which point your cover is blown. I would think that this facet of any proposed guidelines would be most beneficial to the process and yet unattainable at the same time.

There is not a course in the world that accepts raters that wouldn't accept a well traveled gentleman who knows how to ask in a proper manner.

[Matthew Sander]

What is the purpose of the rating process?

Adam,
I'm assuming they are not evaluated by anyone other than the particular magazine for which they work,  The food critic outline was generic for critics working for all magazines.  Also, Adam, for those of you raters that are not part of the segment I am discussing, I think it is difficult for you guys to imagine what is seen out there sometimes.  But as an example, I have had friends who were raters come to a club where I am a member and tell the pro they were a rater expecting not to pay and yet having no reason to rate the course or write it up.  They came back several times that year and did the same.  I never let them know I paid for each time this happened.  I'm not saying the guy is a bad rater but under the food critic guidelines this would never happen.  

Mike,

If this behavior was reported to the powers that be at the publication, you're friend's gig as a rater would disappear quite quickly. You may not be saying the guy is a bad rater, but in fact, he is a bad rater.

[John Kavanaugh]

What is the purpose of the rating process?

Adam,
I'm assuming they are not evaluated by anyone other than the particular magazine for which they work,  The food critic outline was generic for critics working for all magazines.  Also, Adam, for those of you raters that are not part of the segment I am discussing, I think it is difficult for you guys to imagine what is seen out there sometimes.  But as an example, I have had friends who were raters come to a club where I am a member and tell the pro they were a rater expecting not to pay and yet having no reason to rate the course or write it up.  They came back several times that year and did the same.  I never let them know I paid for each time this happened.  I'm not saying the guy is a bad rater but under the food critic guidelines this would never happen.  

Mike,

If this behavior was reported to the powers that be at the publication, you're friend's gig as a rater would disappear quite quickly. You may not be saying the guy is a bad rater, but in fact, he is a bad rater.

I assure you that is not true.

[BCowan]

Mike

    I don't think anonymity is import with Golf rating as it would be with Food, they aren't going to drastically improve the course that morning and service is more important at restaurant than a great golf course imho.  A cook could know to focus more on the dish if rater was know prior or during.  I thought Links Mag. had the rankings the best that I have seen.  Is history one of the criteria for GD?  It would be great to do a gem rating one with certain criteria.  I do think raters should pay to play or how about just walk and rate no play?

Mac

   To branch off your ?'s, a selection of best set of greens, best bunkering, most playable for widest group of players (pro's to 30 handi), best ''bones''.  Word of mouth blog ranking would be cool vs big name published mag?

[Matthew Sander]

What is the purpose of the rating process?

Adam,
I'm assuming they are not evaluated by anyone other than the particular magazine for which they work,  The food critic outline was generic for critics working for all magazines.  Also, Adam, for those of you raters that are not part of the segment I am discussing, I think it is difficult for you guys to imagine what is seen out there sometimes.  But as an example, I have had friends who were raters come to a club where I am a member and tell the pro they were a rater expecting not to pay and yet having no reason to rate the course or write it up.  They came back several times that year and did the same.  I never let them know I paid for each time this happened.  I'm not saying the guy is a bad rater but under the food critic guidelines this would never happen.  

Mike,

If this behavior was reported to the powers that be at the publication, you're friend's gig as a rater would disappear quite quickly. You may not be saying the guy is a bad rater, but in fact, he is a bad rater.

I assure you that is not true.

JK,

I probably shouldn't have responded with such blanket certainty. I am only aware of the ethics guidelines at a couple of the publications. The actions mentioned above are strongly prohibited by those guidelines (on several levels), but I guess the presence of the rules and their enforcement could be two separate things. I'd like to think they matter, but it wouldn't be my first foray into naivete.

[John Kavanaugh]

There is nothing wrong with forgiving a rater for making a mistake. Who knows, the next one you get may be worse.

[Tom_Doak]

Enforcement of ethics for raters at most publications is spotty, if not token.  But to be fair, a lot of times the behavior does not get back to the magazines, because no one at the club wants to be the guy who is reporting on them -- because it's assumed that no action will be taken.

Mike:  These guidelines would be great, but don't you think they are pie in the sky for the golf business, nationally and internationally?  Many raters are somewhat involved in the golf business, and are used to being comped for reasons other than being a rater.

I do not think paying for all your golf is the key to having a worthwhile viewpoint.  Just look at all the people on Golf Club Atlas who let the price of the green fee (or the price of a hot dog) set the tone for their whole experience!  It's fine with me if they want to let that sort of stuff influence whether they have a good time, but I don't think it should be allowed to influence their opinion of my architectural work ... or yours.

To me, the guys who should be kicked off the rating panels are the ones who think it all revolves around THEIR vote.  ["I'm responsible for your course being rated," i.e., you owe me.]  I've seen that several times, and it never fails that the guys who think so are the people whose opinion I value least.

[Mike_Young]

What is the purpose of the rating process?

Adam,
I'm assuming they are not evaluated by anyone other than the particular magazine for which they work,  The food critic outline was generic for critics working for all magazines.  Also, Adam, for those of you raters that are not part of the segment I am discussing, I think it is difficult for you guys to imagine what is seen out there sometimes.  But as an example, I have had friends who were raters come to a club where I am a member and tell the pro they were a rater expecting not to pay and yet having no reason to rate the course or write it up.  They came back several times that year and did the same.  I never let them know I paid for each time this happened.  I'm not saying the guy is a bad rater but under the food critic guidelines this would never happen.  

Mike,

If this behavior was reported to the powers that be at the publication, you're friend's gig as a rater would disappear quite quickly. You may not be saying the guy is a bad rater, but in fact, he is a bad rater.

Matthew,
I understand what you say but my friend is not a bad rater.  He is one of the better ones.  He is objective and does not wet his pants when he meets TD or BC not does he smirk and tell a Rees he has no clue.  He did what he did out of habit.  I paid for him because the proper thing to do is pay for your guest and IMHO he should have never asked.  When I first knew these things were out of whack was when I went to play HDunes in Fla and myself and an asst supt had to pay full fare while the 2nd asst golf pro and the same rater friend were comped.  I paid for myself and the asst supt and played.  The rater " in a subtle " way checked to see if his lunch was included.  The course had not asked to be rated.  We need guidelines like the food industry.  

[Matthew Sander]

Mike,

Understood. I agree with your pov for the most part. There is a certain amount of skepticism and eye rolling directed at raters and the entire process. The icky situations we've all heard about regarding questionable rater behavior do the process no favors.

Like many situations, a big step in the right direction would be the enforcement of existing rules that apply to the very examples you've mentioned.

[Sean_A]

I do not think paying for all your golf is the key to having a worthwhile viewpoint.  Just look at all the people on Golf Club Atlas who let the price of the green fee (or the price of a hot dog) set the tone for their whole experience!  It's fine with me if they want to let that sort of stuff influence whether they have a good time, but I don't think it should be allowed to influence their opinion of my architectural work ... or yours.

I guess you have to trust them to be reasonable.  Probably about as much as you trust raters to fall for the wow factor of a site and bunkers etc.  Archies aren't innocent in this rating scam, they play up to raters just a clubs do. 

Ciao

[Mike_Young]

Archies aren't innocent in this rating scam, they play up to raters just a clubs do. 

Ciao

Sean,
A few may not be but so often the architect is viewed by the outside as being the promoter when in fact it is the developer promoting his project thru the architect.  For the sigs, that why they hired them anyway. 
The few times I have had a rater come at me with the " I am Mr. Rater, jump" attitude, I will bite my tongue for at least 5 minutes before I do all I can to piss him off. 

[Carl Johnson]

Mike, do you mean the golfers who do junk for the magazines, or the the raters who do the official course slope and course ratings for handicap purposes?  I've known both, but the former are irrelevant, in my opinion.  That's a game to get  these guys on the courses.  My concern would be with the latter.

[Mike_Young]

Enforcement of ethics for raters at most publications is spotty, if not token.  But to be fair, a lot of times the behavior does not get back to the magazines, because no one at the club wants to be the guy who is reporting on them -- because it's assumed that no action will be taken.

Agree....

Mike:  These guidelines would be great, but don't you think they are pie in the sky for the golf business, nationally and internationally?  Many raters are somewhat involved in the golf business, and are used to being comped for reasons other than being a rater.

I think the ones who are in the business and are used to being comped are , in most cases, not the ones I am griping about.  (I saw WU, president of a major ball and club manufacturer walk up to the counter in a shop in Cabo one day and pay for himself and his son and never say who he was.)  Those are the guys you really want to comp....  A few of the guidelines may be pie in the sky but they could be modified to fit.  The one single item that I never get a good answer for is paying green fees.  Give me one reason why the green fee should not be paid by a rater.  And if The course being rated thinks it needs to comp in order to bring the rater to his course so that he can obtain the mandatory number of reports for inclusion , that should be stopped also.  One solution would be for the publications to reimburse the rater or writer upon receipt of the requested rating form etc. 

I do not think paying for all your golf is the key to having a worthwhile viewpoint.  Just look at all the people on Golf Club Atlas who let the price of the green fee (or the price of a hot dog) set the tone for their whole experience!  It's fine with me if they want to let that sort of stuff influence whether they have a good time, but I don't think it should be allowed to influence their opinion of my architectural work ... or yours.

Yeah..that hotdog freaked them out didn't it?  I'm not saying paying for all of ones golf is the key to a worthwhile viewpoint.  I'm saying that complimentary fees should not be the norm.  So often the rater presents himself at any club he enters even when he is not their to rate.  As you mention earlier the average golf professional would just as soon comp him as have to deal with it.

To me, the guys who should be kicked off the rating panels are the ones who think it all revolves around THEIR vote.  ["I'm responsible for your course being rated," i.e., you owe me.]  I've seen that several times, and it never fails that the guys who think so are the people whose opinion I value least.
You have said it better than myself...it also goes for the few whack job writers out there that consider themselves career changers...