Nice Article on Cabot St.Lucia project

[Tony_Muldoon]

So David has 'heard' that Muirfield is too small for the Open? 
New one to me, I've been 'reliably' informed that was Lytham's fate.




Nice article. Can't help but think these ratings are the driving force behind green fee inflation.


Note written while DJ was posting.

[Simon Holt]

Steve,


You're wandering into some pretty murky territory here.  Who is the industry insider that you don't have the nerve to name?  I don't think it is particularly fair to leave it to the imagination - it is how people's reputations can be damaged.


Nae stones, as I'd say here in Edinburgh.


I can only speak for myself, but unless I'm not getting emails, none of the GOLF panelists have had any comms on the list since we entered our ballots.  I didn't know the results until I saw David's post, which as a side-note, is a shame.  By coincidence, Paul Rudovsky sent us an update on his spreadsheet the day the world list "came out"....and even he didn't have the new GOLF world list on his update.


Like a good number of the panel, I'm good friends with some of the 8AM guys as we enjoy playing golf together; I actually think you'll find they are particularly harsh when voting, but other than throw away comments I don't know how other panelists vote on more than a handful of specific courses.  Do you? 


I think you'd enjoy talking about courses with the guys far more than you would casting aspersions on their character.  Especially whomever it is that you are suggesting skews votes.  That's just not on, unless you can show us all something.  It's not in the too distant past that courses were on the list that we all scratched our heads at.  Thankfully those days seem to be behind us, but again, maybe you know something we don't.


The lists are always fun.  I like this one.  I was gutted to see De Pan drop out, happy to see Machrihanish back in and as a big fan of Deal, I'm chuffed for the members there, even though it's not quite in my personal top100. 


I've been fortunate enough to see most of this list so I'm eager to see the ones I haven't, but a couple of them are particularly remote.  The air miles might get a kick-in this coming year! 


Also excited to see Te Arai North in a couple of week's time, so it'll be fun to revisit some old favourites like Paraparaumu on the same trip.


I'd be far more interested in hearing about what courses you're excited to see this coming year, rather than this other nonsense.

[Paul Rudovsky]

Going back to the "issue" of courses that have not opened yet (Cabot St Lucia), frankly I do not understand why the concept of "preview" rounds by raters is questionable (maybe because I am a rater  ). 


But seriously...from a rating standpoint, it only matters what the individual raters think...the lists are based on statistical analysis of our evaluations and/or order of preference among a list of candidates.  The views of non-raters (for better or worse) do not figure into the results for courses played before or after opening. 


And in fact...I would expect that allowing raters on a course before opening might lower a course's rating (in general) as there would a greater chance that the course would not yet be fully grown in.  So if a course's managers and/or owners choose to allow us to see it in preview rounds, it seems highly doubtful to me that this would enhance its ranking...and its might well lower it.

[Ally Mcintosh]

Going back to the "issue" of courses that have not opened yet (Cabot St Lucia), frankly I do not understand why the concept of "preview" rounds by raters is questionable (maybe because I am a rater  ). 


But seriously...from a rating standpoint, it only matters what the individual raters think...the lists are based on statistical analysis of our evaluations and/or order of preference among a list of candidates.  The views of non-raters (for better or worse) do not figure into the results for courses played before or after opening. 


And in fact...I would expect that allowing raters on a course before opening might lower a course's rating (in general) as there would a greater chance that the course would not yet be fully grown in.  So if a course's managers and/or owners choose to allow us to see it in preview rounds, it seems highly doubtful to me that this would enhance its ranking...and its might well lower it.

Paul, I have no issue at all with preview rounds for raters…


…However, there is also zero doubt that preview rounds are aimed squarely at the understanding that a certain sect of golf raters just like to be first with the scoop.


There is a huge swathe of the self-proclaimed golf course “connoisseur” sect who are the equivalent of the too-cool-for-school indie kid; the ones who when a band becomes well known, are the first to remind you that they saw them in a shed with only 25 people in attendance, long before they even released their first EP. (More likely, they have already told you many times about this band just to prove that they are more on the ball than you with new, brilliant, music). In golf, they are usually age 30’s-40’s (music a little younger). It’s a whole new demographic.


Therefore, new builds are almost always overrated. The trick for any rater who wants to shout for a certain course is to stay involved for enough rankings (say 10 to 20 years) to ensure that his favoured course sticks around… after that, it becomes the norm in the wider rater group consciousness; and is subsequently less likely to drop like a stone.

[Thomas Dai]

……the equivalent of the too-cool-for-school indie kid; the ones who when a band becomes well known, are the first to remind you that they saw them in a shed with only 25 people in attendance, long before they even released their first EP. (More likely, they have already told you many times about this band just to prove that they are more on the ball than you with new, brilliant, music). In golf, they are usually age 30’s-40’s (music a little younger). It’s a whole new demographic.

Brilliant Ally, brilliant!!
Reminds me of Monty Pythons Four Yorkshiremen sketch.
Atb

[Michael Wharton-Palmer]

Ally..👏👏👏👏

[James Reader]

But seriously...from a rating standpoint, it only matters what the individual raters think...the lists are based on statistical analysis of our evaluations and/or order of preference among a list of candidates.  The views of non-raters (for better or worse) do not figure into the results for courses played before or after opening. 

You say the lists are based on statistical analysis of evaluations. In the case of this Golf list (based on Ran’s description of the methodology used) it’s a fundamentally flawed statistical analysis.
[color=rgba(26, 26, 26, 0.7)]For the newly released 2023-24 World list, each panellist was provided a ballot that consisted of 504 courses globally. He or she was given seven months to complete it. Beside the list of courses were 11 “buckets,” or groupings. If our panellists considered a course to be among the top three in the world, they ticked that first column. If they believed the course to be among Nos. 4-10 in the world, they checked the next column, followed by 11-25, 26-50, and so on out to 250+ and even a column for remove.[/color]
[/color][color=rgba(26, 26, 26, 0.7)]Points were assigned to each bucket; to arrive at an average score for each course, we divide its aggregate score by the number of votes. From those point tallies, the courses are then ranked accordingly. It is an intentionally simple and straightforward process.’[/color]


“Simple and straightforward” it may be but it just isn’t an appropriate methodology to arrive at a 1-100 ranking.  As a simple illustration, if 19 people think Cape Wickham is the 80th best course and one thinks it’s the 101st, and the same 20 people all think Cabot St Lucia is the 99th best course, Cabot would be ranked higher (it would have 20 appearances in the 76-100 bucket v 19 in 76-100 and 1 in 101-125). In fact it would still be ranked higher if more people had seen Cape Wickham and the figures were 99:1.


Quite apart from all the other arguments, you just can’t sensibly create an aggregated ranking which is more granular than the basis on which you’ve asked people to produce their own individual ranking.

[Paul Rudovsky]

Ally/Thomas/Michael—

I would point out that the primary question I was addressing is with regard to preview rounds by raters (and others).  Yes…some who do preview rounds are for sure the ones who want to be the first on their block to play a new or redone course, drive a new car model, or buy a new iphone.  By the same token, there are also others who do not form their own opinions and wait to hear the other views and base most of their conclusions on what they hear or read.  Take your pick, but I will stick with the folks who form their own opinions. My actual sense is that we all tend to base our views on a mix of both…the mix can differ dramatically (90/10 is not equal to 10/90).

I think the use of preview rounds has grown because today it is a rater’s best opportunity to gain access…after opening day the course is often flooded with members or daily patrons.  Side effect from COVID.

I do believe there is an issue that we commonly call “recency (spell?) bias” but I think that is associated with several factors, including that they are fresher in one’s mind.  No question there are lots of examples of courses that shoot up when first built (or renovated) and then fall in the ratings in subsequent years…but the causes may be multiple.  Some examples of other cause are (1) poor upkeep and maintenance, (2) the impact of newer courses coming onstream and other older courses being renovated…remember that ratings are relative to other courses.  And BTW, I have given very low ratings (relative to other rater ratings) to a number of courses that I have played “early” and would bet that is true for most of us.

I am not surprised that architects do not like the concept of raters.  Few like to be “reviewed”…and raters do not like being reviewed either.  But the concept of waiting 10-20 years is a bit silly…and I would ask if any of you owned a golf publication…would you do rating list where only classic courses that have not had a restoration/renovation in say 5 years would be eligible.  Don’t think such a list would get much viewing.

James—

I am very aware of the concept that the results of a survey should not be more granular that the original responses.  But that is a question regarding ratings in general…and has literally nothing to do with preview rounds.  I do find it interesting that you use as an example as situation that I would bet has never come close to happening (especially since ratings with potential conflicts of interest are either not allowed or carefully reviewed)…so I do not agree that your truly outlier example shows that the GOLF process is flawed.  And I would suggest that if you think the grandular rule should dominate…you are free to do something simple…group the GM list as our ballot groups its top 100 positions (1-2, 4-10, 11-25, 26-50, 51-75 , and 76-100).  But the underlying truth is there still will be a #51 in the 5^th group that “perhaps” should be in the 4^th group…and remember that the difference between “25-50” and “51-75” is much greater than the difference between #50 and #51.

[Ally Mcintosh]

Ally/Thomas/Michael—

I would point out that the primary question I was addressing is with regard to preview rounds by raters (and others).  Yes…some who do preview rounds are for sure the ones who want to be the first on their block to play a new or redone course, drive a new car model, or buy a new iphone.  By the same token, there are also others who do not form their own opinions and wait to hear the other views and base most of their conclusions on what they hear or read.  Take your pick, but I will stick with the folks who form their own opinions. My actual sense is that we all tend to base our views on a mix of both…the mix can differ dramatically (90/10 is not equal to 10/90).

There are people who know just enough to be confident in forming their own opinion and are then pretty militant at selling that opinion as the only viable one… It is surprisingly easy to become Ellsworth Toohey in a niche area like GCA…


(Please note that in no way am I referring to you).

[Peter Sayegh]

I think the use of preview rounds has grown because today it is a rater’s best opportunity to gain access…after opening day the course is often flooded with members or daily patrons.  Side effect from COVID.

Wow.
God forbid raters should deal with the flood of "members and daily patrons" to gain the "best opportunity."

Sickening.

[Jeff Schley]

Witnesses say the 2nd gunman on the grassy knoll was a rater.

[Paul Rudovsky]

Witnesses say the 2nd gunman on the grassy knoll was a rater.

Jeff--

By far the best post on this thread!

[Tom_Doak]

“Simple and straightforward” it may be but it just isn’t an appropriate methodology to arrive at a 1-100 ranking.  As a simple illustration, if 19 people think Cape Wickham is the 80th best course and one thinks it’s the 101st, and the same 20 people all think Cabot St Lucia is the 99th best course, Cabot would be ranked higher (it would have 20 appearances in the 76-100 bucket v 19 in 76-100 and 1 in 101-125). In fact it would still be ranked higher if more people had seen Cape Wickham and the figures were 99:1.


Quite apart from all the other arguments, you just can’t sensibly create an aggregated ranking which is more granular than the basis on which you’ve asked people to produce their own individual ranking.

Your second point may have merit, but the example you've cited before that is ridiculous . . . it's just ridiculous to think that many of these raters have broken it down to what they think is the #80 course vs. the #99, or that if they did, that you should take them seriously at all.


I have not seen Cabot St. Lucia, but I was slightly surprised that it rated that low, with all of the hype surrounding it and the obviously spectacular setting.  [Mike Keiser does not often "hint" that a new course might be the best course in the world.]  Maybe someday I'll actually have time to get there.  Probably not this winter, but unlike the magazines, I'm in no hurry.

[Wayne_Kozun]

Tom - speaking of ratings, any update on the final volume of the CG? 

[Michael Morandi]

How did UK Golf Guy scoop the magazine itself?

Someone found a copy of the magazine and sent me a photograph of the list. It was all quite mundane really.

I wrote some thoughts about the ranking here

https://www.ukgolfguy.com/golf-blog/2023-golf-magazine-world-top100

I hope they put it on their website soon as I'm interested to read Ran's thoughts on some of the movements compared to my conjecture!

In your blog you note that Kyle Phillips only has one course in the top 100.  He certainly deserves a ton of credit for the work he did at CalClub.  I’d say he has two in the top 100.

[Mike Hendren]

Do the developers of new courses conduct voir dire of prospective raters to select the 10 most likely to provide favorable ratings?  Surely word gets around who’s hot to trot.


In my 15 years as a GW rater nobody ever sent the private jet for me.

[Mike Hendren]

The Big Ten are vulnerable to being wined, dined and hot boxed during the preview experience.  Surely their rating would feature an appreciation premium.


I was asked to approve a construction loan for a Margueritaville Hotel in downtown Nashville.  The developer invited me to the grand opening of a MH in South Florida where I would get to meet Jimmy Buffett and see him perform in an intimate environment.  I’m a huge Buffett fan but declined the invitation knowing fully well I would not approve the loan for the Nashville project.


Friedman summed it up: “there’s no such thing as a free lunch.” 

[Paul Rudovsky]

Mike--


There is a huge difference between your loan example and rating a course.  In your example, the developer would obviously know if the loan application was approved.  In the case of rating...there is no way for the club/course involved to see my rating (unless they have paid off the administrator of the magazine's panel).  My stored sense is that bribery only works when there briber can determine if the bribee did what the briber expected.

[Tom_Doak]

Do the developers of new courses conduct voir dire of prospective raters to select the 10 most likely to provide favorable ratings?  Surely word gets around who’s hot to trot.

In my 15 years as a GW rater nobody ever sent the private jet for me.

I've been on a few private jets over the years, but never as a rater or even as a contributing editor to GOLF.


It's not at all hard to figure out whose work the panelists like best.  For example, if one of my clients were to be interested in a good ranking, even you know they probably shouldn't invite Darius Oliver to fly over and play right now.      


But you don't have to do think about it; the people who like my work best are going to be the first ones to seek out whatever I've just done.  Indeed, that's part of the reason success begets more success . . . the successful guys have more fans ready to weigh in.  [Note, though, that doesn't explain why some designers' courses move up in the list over time, while others' designs move down.]

[David Jones]

How did UK Golf Guy scoop the magazine itself?

Someone found a copy of the magazine and sent me a photograph of the list. It was all quite mundane really.

I wrote some thoughts about the ranking here

https://www.ukgolfguy.com/golf-blog/2023-golf-magazine-world-top100

I hope they put it on their website soon as I'm interested to read Ran's thoughts on some of the movements compared to my conjecture!

In your blog you note that Kyle Phillips only has one course in the top 100.  He certainly deserves a ton of credit for the work he did at CalClub.  I’d say he has two in the top 100.

Hi Michael. Where do you draw the line? Does Hanse get LACC and Tom Woodhall Spa, OCM Victoria? I think it's hard to do. I did get to see Cal Club earlier this year and when it was explained to me Phillips had done I was really impressed so I understand your pov.

[David Jones]

Do the developers of new courses conduct voir dire of prospective raters to select the 10 most likely to provide favorable ratings?  Surely word gets around who’s hot to trot.

In my 15 years as a GW rater nobody ever sent the private jet for me.

I've been on a few private jets over the years, but never as a rater or even as a contributing editor to GOLF.


It's not at all hard to figure out whose work the panelists like best.  For example, if one of my clients were to be interested in a good ranking, even you know they probably shouldn't invite Darius Oliver to fly over and play right now.      


But you don't have to do think about it; the people who like my work best are going to be the first ones to seek out whatever I've just done.  Indeed, that's part of the reason success begets more success . . . the successful guys have more fans ready to weigh in.  [Note, though, that doesn't explain why some designers' courses move up in the list over time, while others' designs move down.]

i am probably coming across as very naive here Tom but I'll give it a go.


What if courses couldn't enter the list until after a year of opening? Or maybe new courses need a higher threshold of raters? That would get away from the worries about hand-picking the early ones.


My other idea is even more naive. If someone has any financial interest in a course's success then maybe you shouldn't get to vote on it? As a developer, tournament host, magazine publisher - anyone. And I'm not just talking about GOLF magaine here, I mean them all. Not so sure about members, but they don't really need to if enough other raters have played there.


And then how about architects and shapers? I could be really radical and suggest they don't get to vote on courses built in their professional careers. That's probably too far, as we may lose some of the best raters and often they are harder on their own work than others.


This may purely be about optics but I think optics do count for something.


And finally maybe it would be nice if someone invented a list that didn't have individual position but just brackets. No need to respond on that idea anyone!

[Tony_Muldoon]

Mike--


There is a huge difference between your loan example and rating a course.  In your example, the developer would obviously know if the loan application was approved.  In the case of rating...there is no way for the club/course involved to see my rating (unless they have paid off the administrator of the magazine's panel).  My stored sense is that bribery only works when there briber can determine if the bribee did what the briber expected.

This subject has been widely studied and where outcomes can be measured it works.


 Drug companies wine and dine Doctors.  Some Doctors insist they can accept this and not be influenced. They insist that they have the highest ethical standards.
These Doctors typically prescribe a HIGHER no of the drug companies products who are entertaining them,  than the medical average.  Even when generic alternatives are readily available.


Just one example. 


But of course you may be right and you are the one exception. 


Oscar Wilde said if you don't believe flattery works then you haven't been properly flattered.

[Charlie Goerges]

What if courses couldn't enter the list until after a year of opening? Or maybe new courses need a higher threshold of raters? That would get away from the worries about hand-picking the early ones.


My other idea is even more naive. If someone has any financial interest in a course's success then maybe you shouldn't get to vote on it? As a developer, tournament host, magazine publisher - anyone. And I'm not just talking about GOLF magaine here, I mean them all. Not so sure about members, but they don't really need to if enough other raters have played there.


And then how about architects and shapers? I could be really radical and suggest they don't get to vote on courses built in their professional careers. That's probably too far, as we may lose some of the best raters and often they are harder on their own work than others.


This may purely be about optics but I think optics do count for something.

Maybe I'm naïve, but I think most of these rules are already the case (at least technically speaking). I'm happy to be disabused of this notion if I'm wrong.

[Mike Hendren]

GOLF Magazine only requires ten raters to play a course for it to be ranked.

I remain curious as to which ten of the 115 got the nod. A random drawing perhaps.  It is noteworthy that by comparison it took sixteen men of Tain to craft Glenmorangie.  Heck, even Jesus needed a dozen. I guess less really is more.   

[Tom_Doak]

i am probably coming across as very naive here Tom but I'll give it a go.

What if courses couldn't enter the list until after a year of opening? Or maybe new courses need a higher threshold of raters? That would get away from the worries about hand-picking the early ones.

My other idea is even more naive. If someone has any financial interest in a course's success then maybe you shouldn't get to vote on it? As a developer, tournament host, magazine publisher - anyone. And I'm not just talking about GOLF magaine here, I mean them all. Not so sure about members, but they don't really need to if enough other raters have played there.


And then how about architects and shapers? I could be really radical and suggest they don't get to vote on courses built in their professional careers. That's probably too far, as we may lose some of the best raters and often they are harder on their own work than others.

This may purely be about optics but I think optics do count for something.

And finally maybe it would be nice if someone invented a list that didn't have individual position but just brackets. No need to respond on that idea anyone!

Yes, that was an excellent attempt at naïveté.  I doubt they have any interest in doing any of the things you suggest.  Panelists are asked to list any conflict of interest they have for any course they vote on, but I don't believe those votes are often discounted.


As to architects and shapers voting, they are generally some of the very best-traveled people on the panel, and I used to think our egos sort of held each other in check . . . for example, David Kidd sees some of my courses and I see some of his, and I doubt those are the highest votes those courses get.  But there is also a lot of professional respect there.  I don't think we are manipulative about it, I think we vote the way we honestly feel about the work, but I understand why someone who doesn't care about the topic would of course say that's a conflict.


The actual rule that a course had to be seen by 10 panelists to be ranked was written by me, when GOLF started ranking the courses this way in 1983.  We only had about 75 panelists at the time, and the panel was full of people from around the golf business, most of whom who were not nearly as well traveled as you'd imagine.  [There were fewer than ten panelists who had seen more of the courses on the original ballot than I had . . . and I was only 22!]  Many of the golf business people only went to tournament venues, so even places like Prairie Dunes struggled to get ten votes.  But we also didn't have any fanboys or promoters on the panel, so I thought we could trust the votes we did get.  [Who was I to judge?  They were all pretty well respected.]  Honestly the most visible conflict of interest I saw back then was that certain tournaments paid appearance fees to have Seve Ballesteros and Greg Norman show up, and they happened to like those courses a lot, which is why Huntingdale [for one] was ranked back in the day.


It was also me who suggested putting a handful of guys on the panel who were willing to travel far and wide to help us get some of the lesser-seen courses up to ten votes . . . a contingent that has come to be a much higher percentage of the panel today.  I realized it was going to be a problem the night I was traveling with one of them in Japan, and he took personal credit for "putting" certain courses "on the list".


By the same token, if you raise the quorum of votes needed, that's just going to have a remote course like Cape Wickham whinging that they can't get enough people to King Island, or, alternatively, surreptitiously paying for them to come.  I don't think that is happening much right now -- one such promoter was weeded out -- but these rankings are extremely important to developers and paying for a few individuals to come visit would be a rounding error in their promotional budgets.  Be careful what you wish for.


More importantly, the magazines want to scoop each other.  GOLF DIGEST used to have a waiting period for new courses to make their list; I am pretty sure they dropped the waiting period because the GOLF rankings kept scooping them, and those courses credited the GOLF Magazine ranking in all of their paid advertising.