Consensus vs. Groupthink in Golf Course Rankings

[Tom_Doak]

John Kirk had this thought in the thread on the Golf Digest Best New ranking, which I wanted to separate from that and explore more in depth.  I will start by quoting his full post here:

The stated definition of a Doak 7 golf course is:  "An excellent course, worth checking out if you get anywhere within 100 miles.  You can expect to find soundly designed, interesting holes, good course conditioning and a pretty setting, if not necessarily anything unique to the world of golf."

This comment is tangentially related to the topic at hand, but I thought it was worthwhile to share the extraordinary agreement between the Golf Magazine and Golf Digest Top 100 courses in the U.S.I compared Golf Digest's 2023 top 100 list to Golf Magazine's 2024/2025 top 100 list.For context, there are about 16,000 courses in the U.S., according to the National Golf Foundation.

--  Each list has the same top 8 courses.
--  17 of GD's top 20 are in GM's top 20.
--  29 of GD's top 40 are in GM's top 40.
--  47 of GD's top 60 are in GM's top 60.
--  61 of GD's top 80 are in GM's top 80.
--  74 of GD's top 100 are in GM's top 100.

Not only that, but Golf Magazine's new list includes the Lido, CapRock Ranch and Old Barnwell, which could all conceivably make Golf Digest's next top 100 list.So out of 16,000 golf courses, these two lists with very different evaluation criteria agree on about three-quarters of the top 100 courses. 

How do you explain that?  Why is there such an extraordinary agreement?  It could be groupthink, but my ongoing inner debate about subjectivity versus objectivity is piqued by this result.  I'm willing to concede that "there is no objectivity, only intersubjective agreement built around agreed-upon frameworks for analysis."  But the level of agreement here is astonishing.I guess this is my way of reminding everybody how uniform and refined the evaluation of courses has become
.

[Tom_Doak]

I'm going to take the stand that it is more a matter of groupthink than anyone wants to admit.


I've arguably been watching the rankings of golf courses as long as anyone here, and I have been privy to a fair amount of insider information, between knowing Ron Whitten and Brad Klein and Ran Morrissett, in addition to running the GOLF Magazine ranking for 12 years [10 or 12, can't remember for sure].




There has been a status quo since GOLF DIGEST went from their initial list of "America's 200 Toughest Courses" in 1966 to "America's 100 Greatest Courses" in 1969.  That second list was a bit poisoned by the first, and criteria such as "Resistance to Scoring" and "Shot Values" [whatever they are] were big factors in GOLF DIGEST's advice to panelists.  Of course, how much they factored in is debatable, since there were no numbers being handed out in any of those categories until 1985 . . . but I think it's fair to say that in those days, difficulty was considered an important part of "greatness".


When GOLF Magazine published their own early lists of the Top 50 Courses in the World, and then the top 100, there was something of a pecking order established for the U.S. courses by GOLF DIGEST, but not for the overseas courses . . . that took longer to develop.

[Tom_Doak]

Surely, tastes have changed over time, and the criteria for rating courses has evolved . . . and that is one of my examples of why it's groupthink.


My other example is The Confidential Guide to Golf Courses.  When I wrote the original version of the book in 1987, some of the courses I highlighted as my favorites were nowhere to be found in the ranking lists:


Cypress Point, as I recall, was in GOLF DIGEST's Fifth Ten
Cruden Bay was in nobody's list
Crystal Downs was in nobody's list
Lahinch was pretty low in many people's estimation, for all its blind shots
The National Golf Links of America was in nobody's list [!!]
North Berwick was considered much too short and easy to be a great course
Rye [par 68] was in nobody's list
St. Enodoc [par 69] was in nobody's list
Swinley Forest [par 68] was in nobody's list


Now, you could argue that times have changed, and that people's criteria have changed, but why is that?  How does consensus just slowly change over time?


It changes because of groupthink.

[Tom_Doak]

Groupthink is why when GOLF Magazine started ranking courses from 1-100, GOLF DIGEST had to follow.


Groupthink is why when people in the business thought certain courses like Cypress Point, NGLA, and Fishers Island appearing higher on the GOLF list made it a better list, GOLF DIGEST started re-writing its criteria to boost those courses up its own list.


Groupthink is why Pine Valley stays at #1 . . . even though it is very easy to make the case that it's basically torture for 50+% of golfers.  But most panelists argue that it is SO GREAT that we must discount that . . . because each of the holes is so compelling [if you are good enough].  In the end, all they are telling you is that if you're going to complain on those grounds, you're not good enough to judge, like they are.  God forbid that such a view would be part of the Consensus.  It must be struck down, because Groupthink.


Groupthink is why, when GOLF DIGEST panelists rate courses on seven different criteria, they give the same scores in 5-6 categories and are afraid to go too far above or below the overall number they want to give.


A year ago, someone approached me about starting another, independent rating system for golf courses, and asked how we could make it different.  My first thought was to try to rate courses based on the criteria Mike Nuzzo suggested here a few years back:


Beautiful
Challenging
Fun
[and I thought I might add, Interesting] 




What I realized was that NO course would legitimately score a 10 out of 10 in all four categories.  And if we did that, it would very much call into question whether the vaunted top 10 or top 50 on the other lists are really as far beyond the rest as they've been put up to be.

[Ally Mcintosh]

Entirely driven by groupthink, Tom.


I’d be absolutely fascinated to see what everyone’s individual poll would be if each ranker’s memory was wiped of all media and any knowledge of others’ opinions


It would be laughably different.

[Tom_Doak]

Entirely driven by groupthink, Tom.

I’d be absolutely fascinated to see what everyone’s individual poll would be if each ranker’s memory was wiped of all media and any knowledge of others’ opinions

It would be laughably different.

You could give it a shot yourself, Ally.  If you weren't afraid of being crieth down.

[Ira Fishman]

I am not a rater, but if I were, I doubt that the higher up one goes up the consensus/groupthink lists that I would buck the consensus dramatically. For example, I think that Royal Dornoch is overrated and that Yale and Pasatiempo are underrated, but would I drop RD out of my top 20 or move Yale into it? I am pretty sure I would not go that far. Toward the bottom of the top 100 lists, I would be much more likely to buck the consensus.


I am an iconoclast, but I am an amateur. The influence of others is influential.


Ira

[mike_malone]

This is what I hoped to fight with my rankings topic. If the masses could introduce some different courses for consideration.

[Tom_Doak]

I think that Royal Dornoch is overrated and that Yale and Pasatiempo are underrated, but would I drop RD out of my top 20 or move Yale into it? I am pretty sure I would not go that far. Toward the bottom of the top 100 lists, I would be much more likely to buck the consensus.

I am an iconoclast, but I am an amateur. The influence of others is influential.

Could be the working definition of Groupthink.  I disagree, but I can't disagree too much!

[Ally Mcintosh]

Entirely driven by groupthink, Tom.

I’d be absolutely fascinated to see what everyone’s individual poll would be if each ranker’s memory was wiped of all media and any knowledge of others’ opinions

It would be laughably different.

You could give it a shot yourself, Ally.  If you weren't afraid of being crieth down.

I’m not really up for having my memory wiped though!


One thing is for sure: With no media and no reference points, a load of rankers would not have the first clue about whether a course had ODG pedigree or not. That in itself drives a lot of groupthink. As does Championship history / moments.

[Ira Fishman]

I think that Royal Dornoch is overrated and that Yale and Pasatiempo are underrated, but would I drop RD out of my top 20 or move Yale into it? I am pretty sure I would not go that far. Toward the bottom of the top 100 lists, I would be much more likely to buck the consensus.

I am an iconoclast, but I am an amateur. The influence of others is influential.

Could be the working definition of Groupthink.  I disagree, but I can't disagree too much!

Yep. I think the lists or at least most of the ordering is a product of groupthink, including my susceptibility to it. I have seen a lot of groupthink in my 45 year career so I know it when I see it!


But I also think that there is a drop off in quality once the courses are not in the top, let’s say, 500 (just a raw number).

[Tom_Doak]

One thing is for sure: With no media and no reference points, a load of rankers would not have the first clue about whether a course had ODG pedigree or not. That in itself drives a lot of groupthink. As does Championship history / moments.

I agree about the pedigree and we all know that's a big part of people's thinking. Look at the art world!


I don't understand the part about Championship history.  I have always discounted that and it is really easy to ignore if you want to ignore it.  Great and memorable shots can happen on any hole, whether it's good or bad!  But the fact that some courses are on TV every year and are a million times more familiar than a brand new course is a huge advantage.  [Even so, a lot of tournament courses are absolute trash.]

[Matt Schoolfield]

It seems obvious to me that with any type of artistry, there is fashion. With fashion, there will be trends that become popular and fade, and there will be icons of previous trends that remain. I think it's safe to say most of us still love the Talking Heads or Prince, but maybe we no longer care for A Flock of Seagulls or Wham!.

I think the elitist mindset misses the nuance between design skill and general novelty (I do not mean "elitist" here disparagingly, only that the mindset presupposes is a single vector of "goodness"). I think there is a significant amount of overlap between something being considered "good" and something being considered "interesting." That something being interesting requires at least a non-trivial amount of novelty means that success will require some experimentation. When a novel approach is compelling, we will see dozens of folks iterating on it. Iterations on a theme often exceed the interest of the original, but after a good amount of time it becomes increasingly difficult to surprise, and the trend fades.

This is normal and is a perfectly reasonable outcome in artistic creativity. It also leads to apparent groupthink, which I think is probably better explained by the genuine interest of novelty, following by waning of interest. When interest in a trend wanes, it's often quite clear who created the consensus iterations on the theme. There are two parallel tracks of consensus: novel ideas and best iterations on those ideas. Trends rise and fall because they are interesting and then become boring. If you have an elitist mindset, then this looks like groupthink overtime, but if novelty is part of a complex goodness, then you genuinely have courses that change in their level if goodness over time.

Sure, critics often get the zeitgeist wrong (Psycho, Fight Club, and even It's a Wonderful Life were all panned by critics), but the audience themselves can actually get things wrong too (American Beauty, Love Actually, and Crash [2004] are all movies that most people like, and now have no idea why they liked them). This is difficult to explain if we have a single vector of goodness, but if we have a dynamic model of goodness that is based as much in human experience as it is in the thing itself, the all of these trends and groupthink are trivially explained by certain pieces of are having different impacts on people at different points of time. I fully understand that this is an unsatisfying explanation of aesthetics, but it can still be a useful one, again, if we think of critics as proxies for peoples various and contrary tastes, instead judges of goodness. Even if critics disagree, we can still find critics we generally agree with and use their opinions. We do here, however, lose the benefits of aggregation and/or consensus as a guide.

[Craig Sweet]

Maybe we need term limits on raters?

[Andrew Harvie]

I'd guess a lot of that groupthink comes from not only more people interested in golf architecture, but also fewer barriers of entry for the subject with the internet. To add to that, the media on the subject—be it golden age books or modern media—does sort of advocate for the same principals. It's not like Tom Doak's books are contradicting Mackenzie's writings; there really isn't much opposition to what Ran and others have written on GCA, nor the Fried Egg now. If everyone is digesting the same content in just different packaging, it's not hard to see where the similar thinking is coming from, and it's not like Rees Jones or Desmond Muirhead have books a contrarian can really dig their heels in with (though I would love to read that!).


That's part of the reason I enjoyed Tom Fazio's interview in No. 30 of The Golfers Journal, because it really did seem like he a) believed he was right, and everyone else was wrong, and b) he had a mind of his own on the subject. It's a different opinion, and he dug his heels in on a bunch of topics. That, in itself, is an interesting read because it's much different than what we consume here, or on Fried Egg, or in Harry Colt's writings, etc.

[John Kavanaugh]

If you slobber on yourself like a dog to a bell before you hit the parking lot it’s called classical conditioning. Until we can control the social media photo narratives I don’t see a cure.

[Sean_A]

I think we must accept some degree of groupthink. 1… there are relatively few courses considered for these lists and 2…there are relatively few people rating these small number of courses and 3…previous lists influence future lists.

To me championship pedigree means something (exactly what is more like a tie breaker all other things being equal) because the course can challenge the best players. If we are hoping the best courses are for everyone then we must include elite players.

Ciao

[Ben Sims]

Okay fine. You want guts?


To see the really great places, you have to be hosted. Someone is taking the time to be with a likely stranger. And you’ve been told, as an enthusiast, that this course is the one to which all others should aspire. People you admire also think this course is the greatest. And then you play it. You buy the course history and spend half a mortgage payment in the shop. You read, for YEARS, how good the place is. But this feeling gnaws at you…it’s just not what they all say it is.


Is it me that’s crazy? Yeah probably. I’ll just mark it a 10 cause Brad said my rating gets thrown out if it’s too far off the norm anyway.


Groupthink is powerful!

[John Kavanaugh]

Don’t be part of a group.

[Tom_Doak]

It seems obvious to me that with any type of artistry, there is fashion. With fashion, there will be trends that become popular and fade, and there will be icons of previous trends that remain. I think it's safe to say most of us still love the Talking Heads or Prince, but maybe we no longer care for A Flock of Seagulls or Wham!.

I think the elitist mindset misses the nuance between design skill and general novelty (I do not mean "elitist" here disparagingly, only that the mindset presupposes is a single vector of "goodness"). I think there is a significant amount of overlap between something being considered "good" and something being considered "interesting." That something being interesting requires at least a non-trivial amount of novelty means that success will require some experimentation. When a novel approach is compelling, we will see dozens of folks iterating on it. Iterations on a theme often exceed the interest of the original, but after a good amount of time it becomes increasingly difficult to surprise, and the trend fades.

This is normal and is a perfectly reasonable outcome in artistic creativity. It also leads to apparent groupthink, which I think is probably better explained by the genuine interest of novelty, following by waning of interest. When interest in a trend wanes, it's often quite clear who created the consensus iterations on the theme. There are two parallel tracks of consensus: novel ideas and best iterations on those ideas. Trends rise and fall because they are interesting and then become boring. If you have an elitist mindset, then this looks like groupthink overtime, but if novelty is part of a complex goodness, then you genuinely have courses that change in their level if goodness over time.

Matt:


I am a proponent of the Big World Theory to some degree . . . that is, I am open to a wide variety of styles of design, as long as they are well executed.  I like variety, and I think that any list of great courses should be full of variety.  A top-100 list that has a lot of Seth Raynor courses is as pointless as a top-100 album list that has all of The Beatles' albums on it.  [Even though The Beatles showed a wider range in their short careers than Seth Raynor did.  And, come to think about it, they were both all the rage for only a few years.]


But during my career, I've seen GOLF DIGEST fawn over Jack Nicklaus' work, and slobber over Tom Fazio's, and then fall in love with the work of Mike Strantz, and Jim Engh.  Certainly, the latter two did not spawn a wave of copycats, and their stars fizzled very fast; and you could argue that no one else really tried to emulate Jack or the Faz, either, even though they were quite successful at their thing.  [Maybe their styles depended in part on budgets that nobody else had.]


I do think you are right that "interesting" is often conflated to "good" and that may not really be the case.

[Ben Sims]

Why is it controversial to say that it’s preposterous that only 3 of the best 20 courses in America were built in the last 40 years?

[Tom_Doak]

Why is it controversial to say that it’s preposterous that only 3 of the best 20 courses in America were built in the last 40 years?

Well for one, there would have to be very strong agreement on what the fourth and fifth new courses are.

But I also think that one reason so many of the older top 20 have gone in for expensive makeovers in recent years -- by the same designers who built the courses contending for their spots -- is to be sure they aren't overtaken, and incorporate the trendy features that made the newer courses seem fresh.


EDIT:  Sorry, had to fix the sizing

[Jake McCarty]

Why is it controversial to say that it’s preposterous that only 3 of the best 20 courses in America were built in the last 40 years?

Bingo. And the best were the first.


What I've found being a member at several highly regarded courses.


1)  My friends who are ambivalent about ratings and have a high golf IQ always preferred certain courses of mine. And they were always the same preference.


2) When I would host raters, they would often parrot magazine lists.


So what I learned is that I generally won't host raters as I find their thinking is predictable and uninteresting, even though many of them are super cool and fun people. And I have communicated with several of my clubs to no longer allow rater access unless they appear to be comfortable thinking outside of the box.

[C. Sturges]

isn’t it automatically group think?   aren’t raters given a list of courses to be rated?
my guess is a couple hundred courses in the USA.   so the top 200 are going to be extremely similar.
also for new courses, there are only so many a year.   and courses want to publicity, even the ultra exclusive so they open there doors to raters.   or at least from the sounds of things they do.
and lastly, you get the golden ticket of being a rater(many times after paying lots of $$$) of course you are going to try and play as many of the previous top 100 courses.   
seems it makes it near impossible for not having group think.

[Matt Schoolfield]

But during my career, I've seen GOLF DIGEST fawn over Jack Nicklaus' work, and slobber over Tom Fazio's, and then fall in love with the work of Mike Strantz, and Jim Engh.

Dare I say that the advent of color television may have been what brought interest to the design choices of Nicklaus' cohort? The "make it as green as possible and put trees everywhere" may not make for the best golf, but it certainly pops on TV. While this won't having a lasting effect, when the paradigm is focused on a different medium, I think it's understandable that artistic values might be different.

To follow one controversial statement with another, I dare say that the dramatic increase in equipment in both distance and accuracy in the late 90s may have sparked much of the current interest in a return to links-style play, with a focus on wider, rumpled fairways and wind, since much of the interesting challenge of the more penal designs of the 70s and 80s is neutered by modern clubs.

I obviously have my preferences, and my neurotic brain cares much more about optionality than fairness, but I have plenty of friends who grind it out at the range every week who think the randomness of windy linksland is dumb.

[Tags:]

Tags: