Shackelford on GD Rankings...

[Ouch!]

Don't believe the list hype
By Geoff Shackelford
Contributor to Shark.com

The Associated Press carried the news on its wires. Even ESPN's SportsCenter reported that in a shocking twist, Pebble Beach Golf Links has dethroned Pine Valley as the No. 1 course in the United States according to Golf Digest's "America's 100 Greatest Courses."

Was it news? Or silly hype?

The magazine's highly anticipated bi-annual top 100 list and accompanying state-by-state register of greatness tries to improve each time out by unveiling more categories to identify top-notch courses. Noble maybe, but not only has the effort proven fruitless, the out-of-control mess known as the "Top 100" has become such a disaster that a small group of people behind the scenes have to create confusion by adding more categories and bonus points just to save the magazine from embarrassment.

Some of attempts to mask panelist oversights are comical, other panelist blunders are foolish and some are downright irresponsible in trying to identify "great" courses. Consider that Bethpage Black, a frequent member of the Top 100 club over the years, dropped off the list in 1999, only to reappear in 2001 at No. 46. How can this happen?

"We removed it in 1999 while it was being remodeled by Rees Jones in preparation for the 2002 U.S. Open," wrote Digest architecture editor Ron Whitten. "Frankly, the removal was our mistake, but our panelists bailed us out, giving Bethpage such high marks that is springs back onto the list at a lofty 46th place."

But if the process is so good at identifying greatness, how can a course undergoing a simple remodel fall off the list? This is not the first renovation in the history of golf, so how does the process allow this to happen?

Confusion. Consider how the process for identifying greatness works. Or doesn't work.

The Stadium Course at PGA West debuted on the Top 100 list, then fell off a few years later.
Eight-hundred panelists described as "publicity-shy" single-digit handicappers (hacks with an eye for architecture are not welcome) weigh in on new and old courses. The golfers are selected because they are well connected or because they carry a low handicap or because they travel a lot. The concept of evaluating architecture still hasn't been mentioned as a pre-requisite for membership.

The panel looks at eight categories, issuing a 1 to 10 score. The well-defined "Playability" category used to count for public golf, but somehow it's not relevant to Top 100 greatness. The actual categories used are "Shot Values," "Resistance to Scoring," "Aesthetics," "Design Variety," "Memorability," "Conditioning," "Ambience" and "Walkability."

The last two categories are new for 2001 and are included in the infamous "Tradition" score. That part of the scoring equation has become legendary because of its arbitrary ability to correct the Top 100 as the editors see fit. For an interesting exercise, check out the 2001 Top 100 list without the "Tradition" score added. It's interesting to see how much ambience, walkability and tradition move the list around, especially in the top 10.

Several things about the 2001 list reinforce doubts about the credibility of the rating process even though the efforts to cover so many aspects of a design seem well-intentioned.

Beside the fuss over Pebble Beach surging to No. 1 thanks to a new $9 million par-3 and better "ambience," the Digest list reveals a disturbing trend: panelists fall in love with certain new courses, only to cruelly turn on those courses just a few years later. It is a dangerous trend because money is subsequently put into "fixing" the course to get it re-listed even though nothing has changed since the place opened. Or worse, superintendents and managers lose jobs because the course drops off the list. All because the panel realized it got a little too excited when the course first opened and failed to notice if any decent architecture was present.

Consider these layouts from the 1993 Golf Digest Top 100 and their standing today:

COURSE 1993 TOP 100 RANK 2001 TOP 100 (2001 STATE RANKING)
Oak Tree Golf Club No. 52 Not Ranked (No. 3 in Oklahoma)
Desert Mountain (Renegade) No. 53 Not Ranked (No. 8 Arizona)
Troon Golf Club No. 55 Not Ranked (No. 7 Arizona)
Haig Point No. 58 Not Ranked (No. 9 S. Carolina)
Barton Creek (Fazio Foothills) No. 60 Not Ranked (No. 7 Texas)
Old Marsh No. 78 Not Ranked (No. 13 Florida)
Sherwood No. 79 Not Ranked (No. 16 California)
Wynstone No. 85 Not Ranked (No. 22 Illinois)
Desert Mountain (Cochise) No. 87 Not Ranked (No. 15 Arizona)
Kaui Lagoons (Kiele) No. 88 Not Ranked (No. 6 Hawaii)
PGA West (Stadium) No. 93 Not Ranked (No. 25 California)
Wild Dunes (Links) No. 100 Not Ranked (No. 7 South Carolina)

How can a course go from a Top 100-caliber design to one that is barely ranked in its own state eight years later, such as Wynstone or the Stadium course at PGA West? If anything, those courses have matured, not deteriorated. But because the Golf Digest panel is so easily infatuated by the latest multimillion-dollar "experience" instead of focused on the integrity of the design, the army of panelists keeps making questionable selections that they quickly back down from in short time.

Which brings us to this question: if Golf Digest rated the Top 100 films, how would they do it?

According the system they use now, the cleanliness of the theater floor would have as much impact on a film's rating as the character development. The attitude of the ticket taker would mean as much as the plotline.

The timeless nature of Pebble Beach and Pine Valley stems not from having the best accoutrements and experience, but from their enduring design strategy and the character of the architectural features.

Sure, the experience is great at those courses and there is no doubt they are special. But regardless of the grandeur of the locker room, the caddy uniforms or the food in the grille, it is strategy and design character that carry certain courses to the top.

It's the same way timeless films rely on a unique plot, an emotional theme and wonderful acting, and not on the cleanliness of the theater. Yes, a bad theater can take away from your enjoyment of a film, but would a film critic hold that against the movie in his review? And wouldn't the critic be shrewd enough to look past the burnt orange drapes if he was watching Citizen Kane?

Conversely, the focus on golf experiences and other non-essential elements has me convinced that Golf Digest would rank "event pictures" like "Independence Day" or "Mission Impossible 2" in the top 100 films of all time. These summer B movies, which rely on special effects to cover a thin story, would make a big debut, only to drop off the list once the panelists went back and recognized what they should have the first time: that they lacked substance, character and a story.

Meanwhile, the story told by a Cypress Point or a Merion is brilliant not only on first inspection, but on the 50th time around too. They are "The Godfathers" and the "Citizen Kane's" of golf. These courses are not great simply because they are old and famous. They are timeless because their strategic, charismatic architecture holds up over time. They tell a great story that every generation can relate to.

So until the majority of panelists stop judging a course based on their own personal experience (or how well the staff treated them or what score they shot), new courses are going to continue to make big ranking splashes, then quickly fade like bad summer movies. The Digest editors will have to keep fudging the numbers as long as the panel emphasizes the latest, greatest experience -- or worse, their own games -- over the architecture of the course.

[ted janeczek]

geoff: you haven't left much else to say nor have you many friends at gd!! thanks for sharing your insights. the people i know who are gd panelists don't know a thing about architecture, are low handicappers and are"influential" in the world of golf. amen.

[Ted_Sturges]

Nice work Geoff.  Accurate, and a funny read.

TS

[Geoff_Shackelford]

Thanks Ted and Ted. This appeared back in May when the rankings came out, glad "Ouch" could find it on shark.com (nice site, hard to find old stuff).

[Scott_Burroughs]

Here's some other GD ranking anomalies:

NGLA, not ranked from 1969-1984, then started at #56, now #16.

Shadow Creek, debuts at #8! 8-10 years ago, now #31.

Shoal Creek, '85-'86, #14!, now #68.

Valley Club, not ranked until 1995, now #59.

Wade Hampton, debuts '91-'92 at #17!, now #37.

Long Cove, '85-'86 at #19, now #93.

For many years before I was enlightened, this was the only list around (that I knew of) and I thought it was The Golden List of Lists, like many others believed and probably still believe.

[aclayman]

I found the aricle to be curiously "Psychic" No sooner did I start to think about the Oscars, did you assimilate film. I felt the subject was kinda self defeating but the point is well taken and everyone should make their own opinions about what they consider to be important when choosing to golf at a certain course.

Now to find out that this was writtten back when all the hoopla about the rankings was in full gear. Have you had any "real" feedback from the panelists you made reference to?

Was it your hope, when writting that article, to educate the well connected, low cappers about the underlying fabric that makes their golf what it is today?

Adam

[John_Conley]

I've said it before and will repeat... if you keep changing the criteria it will yield an ever-changing result.

Many on this board bash the rankings, but if you don't agree with the criteria being measured you will inevitably reach different conclusions.

It used to be the hardest 200 courses and now has evolved to place a huge premium on Tradition, a category courses like Bandon Dunes and Sand Hills are virtually ineligible for.  (Side note - the Sand Hills course is the most "traditional" design if you hold The Old Course at Saint Andrews as the model of how the game evolved.  Yet it gets no points.  Different interpretations of "Tradition".)

Debate spurs interest and interest fuels magazine sales - which is what these are for anyway.  

Great article by Shaq that points out these foibles and explains how they happen.

[Matt_Ward]

Enjoyed reading the comments.

Now a few of my own.

I am a GD panelist / age 44  ... have been since 1984. I take great pride in studying the works of all types of architects -- those from the past and those practicing today.

I pay my own way when I travel and I make it a point to reach out to those I know in golf to give me insight on different courses from all parts of the United States.

Even though I am a panelist I too have disagreed with the results of various rankings over the years. I don't believe in points being awarded for ambiance and walking because often these categories are used to prop up old-time courses at the expense of exciting new designs.

I don't knock out superior old-time courses at the expense of the latest "bell and whistle design." I caution people to be sure not to broad-brush all GD panelists. At the same time there are individuals who have an affinity for courses from the 20's and 30's and see nearly everyone one of the them as a "classic." Dogmatic thinking in either case is not really helpful.

I am not "sucked" in to the biggest and most beautiful brochures that tout the newest courses as the greatest thing since slice bread.

Yes, the wherewithal to travel is essential. If you don't constantly make it a point to see what is taking place "in the field" how can you possibly claim to be a national rating panelist? I have played 90 of the current GD listing and I am able to make at least 3-4 major golf trips in the year to various locales depending upon what I hear on the grapevine. My background comes from public golf and I certainly am not from blue blood stock.

I do not belong to any club so as to avoid any possible conflict of interest. I cannot say the same for other panelists. I am also concerned that certain panelists display a regional bias in their voting. I live in the greater NY / NJ metro area and there are some courses in my opinion that are vastly overrated because of the media attention in the area. At the same time there are other courses in the same area that cannot get attention because the spotlight shines so brightly on the same courses year after year.

I just returned from a major golf trip throughout the Midwest and was most impressed with the following course:

Skokie CC (completely updated by Ron Pritchard from the Ross handiwork). A worthy contender in my opinion for Top 100 consideration. The former US Open site (Sarazen's win in 1922) is a unique experience and vastly underrated in the quality of golf offerings in the immediate Chicago area. Great green sites, all bunkering has been updated and the exquisite routing is something all golf lovers should see.

I also had the opportunity to play Shoreacres  (currently rated among the Top 100) and was extremely disappointed. Sadly, there is a tendency to think that everything the old masters designed is a gem. At Shoreacres there are several holes of note but the layout in my mind was ordinary in its overall presentation. Obviously, a majority of other GD panelists believe differently since Shoreacres is among the elite 100. I truly believe someone goofed with the absence of Skokie and another Chicago area course Rich Harvest Links in Sugar Grove, Illinois.

For all GD bashers out there keep in mind that there are panelists who do not agree with the findings that are eventually published.

Clearly, ratings in the final analysis is a subjective exercise. I try my very best to study styles in golf course design and apply a broader picture on how the design can work for all types of golfers. I am not swayed by all the non-related design features such as the grandiose nature of the clubhouse, the type of food served and all the other puffery that goes on. I do judge the golf -- from the 1st tee to the last green. I don't care if the course has a trailer or the Taj Mahal as a clubhouse.

The final published ratings will never satisfy everyone. But please be aware that there are people out there who are GD panelists who love the game no less than the next person and take great pains to be evenhanded in our evaluations.  

[T_MacWood]

Matt
I don't think the article was questioning the panelists love for the game, simply the rankings abiltiy to identify the greatest designs. The rise and subsequent fall of the newest super course is a certainty. The bump given to the US Open venues is also a given. I understand you are but one man among many and that you don't completely agree with results, but not so long ago you said Nantucket was a wonderous design and Bulle Rock was a masterpiece--I wonder if you will feel that way four or five years from now.

[aclayman]

Matt, I have been fortunate enough to have played both Shoreacres and Skokie CC. The former was over 15 years ago, the latter just recently. While my knowledge and awareness about everything has grown I truely had no idea what Shoreacres was until I played it.

My question to you is did your expectations play a large part in you formulating your opinion ?

Have you ever been to Lawsonia? Those cross bunkers at Skokie were miniature compared to those in GreenLake Wi.

[Tom Steenstrup]

aclayman, you mention the Oscars. I couldn't help but remember an article in Premiere magazine about the last 20 years of Oscar winners. They compared the actual winner to a recent online vote and a tally by accomplished filmmakers. Not surprisingly, there was several years where the two votes didn't agree with the Oscar choice, but surprisingly, the popular vote almost always agreed with the filmmakers choices. In essense, time tends to find the best films to be both the critical and popular favorites.

http://www.premieremag.com/premiere/oscars/revisited/intro.html

We can hope that the same applies for golf courses.

Tom

[Tommy_Naccarato]

Matt,
Weren't you the guy that just recently suggested Lost Canyons as a US Open Site?

While I have yet to fully see Lost Canyons, I have in fact played three holes and seen many others thanks to a recent photo shoot for a magazine, and I have to say that I was less then impressed by that comment and it came to mind several times during the recent visit.

Lost Canyons seems to be a good course, however it is not a "GREAT" course. At least in terms of holding a US Open. For me, this statement holds water with Mr. Shackelford's article.

However, I too am impressed by your passion and love for the game, but as a critic, I think that one's standards have to reign supreme as to what is great, good, & bad and how to grade or assign a number to it. Your assessment of the time and money it takes to be a panelist is pretty much spot on, but lets face it, it sure is a nice calling card isn't it?

Please take no offense, it just my take on things.

I have a question for all panelists from all the magazines:

"If your particpation on a magazine panel meant that you didn't get access to actually PLAY, that you were only allowed to see the courses, would you still be a participant?"

[David Wigler]

Tommy,

I know where you are going with you question but it would be impossible.  Asking a golf rater to rate all their courses by just viewing them in the equivalent of asking a food writer to review food based on appearance or a movie reviewer to review movies based on the trailers and press releases.  Many of us have criticized GD for falling in love with the newest clubhouse, best service, most attractive beer cart girls, etc.  Without playing the courses this would be more prevalent not less.  Perfect example - I recently played the Oconee course that Rees Jones did at Reynolds Plantation.  It is a visually stunning golf course.  Without a club in my hand, I would have said it comfortably belonged in top 100's.  It was only in playing it (And trying to think through the best strategic approach to the holes) that I recognized that too many holes played similar and required identical shots and strategy.  Furthermore, some of the most attractive visual features, like the river running through #12 were positioned for aesthetic but wrecked the holes strategy (Subdividing a tight fairway forced a mid iron and eliminated the intended landing area).  To answer your question, I would not rate courses if I could not play them because I would knowingly be incompetent instead of this blissfully ignorant incompetence I now experience.

[Jeff_McDowell]

Dave,

I feel differently about being able to analyze a course while playing.  I can't do it.

I spend too much time thinking about my next shot, and don't have an opportunity to consider different options. And I certainly don't have the opportunity to walk to different areas of a hole to see how it would play different.

[spooon]

Tommy

How can a hack review a course by playing it any better than by just walking the course or reading a magazine.  When you evaluate a design do you drop balls in the fairway where a decent player may hit from or does your evaluation of a course come from the maintenace and location of various drop areas.

[ForkaB]

There is a line of thought in Learning theory that one's ability to do any given taks goes through a cycle based on a 2X@2 matrix on Competency and Consciousness, as follows:

Unconscious/Incompetence--you don't know what to do and you can't do it either.

Conscious/Incompetence--you sort of know what to do, but you still can't do it.

Conscious/Competence--your know what to do and you can do it

Unconscious/Competence--you can do it, but you don't know how you do it.

The latter stage is equivalent to "playing in the zone."

I will leave it to others to speculate where "raters" are and should be on this scale.

PS--The highest level of learning is when, after you have reached a "zone" you realize that you really don't know much at all and you go back to step one again, albeit on a higher plane.

[RJ_Daley]

spooon,

I don't completely buy into your thoughts that a hack can't evaluate a course if playing skills are lacking.  What is essential for the hack to know is variation of design by having seen and played many styles from many parts of the golf world.  That person needs to know history of golf architecture and what goes into routing and building a golf course first to evaluate other courses.  Also, that person has to be an observer of the game as it is played by a cross section of people who really enjoy it.

If that person's playing skills are not that of low handicap to scratch, that person may still play the course and recognise what various shot options are offered from various places tee through green, and can putt with as much observational and analitical skills as required to determine the greens merits, even if they don't putt at competitive putts per round.  Even a hack can tie Tiger Woods on any given hole, but the score really does not matter.  The hack may hit numerous attempts to finally execute the shot that one sees as an option from one point to another relative to assessing the design and quality factor.

I would much rather go down the road that Rich Goodale suggests in obtaining a proper assessment of any given golf course.  To me Rich's thoughts suggest that a complete evaluation isn't very good unless it is made on several levels and on multiple attempts.  I don't think that we would get consistent and well reasoned evaluations of golf courses even by touring pros it they only played these courses once and then submitted their evaluation form.  Look at the example of Bobby Jones and how he hated St Andrews the first time he played it.  Look at Scott Hoch's reported distaste for SA and links in general.  Yet, Jones came to love the old course and understand its merits over repeat times of play.  Whenever low or high handicappers are assessing a course for ratings, if they are submitting assessments after one play, they are all subject to the WOW factor and can never hit enough shots in one round to evaluate a great design past commenting based on first impression.  In a way, first impression as a statement of greatness is contrary to the whole idea of the enduring quality and consistent appeal of a really good golf course, if variety and enduring quality are in deed high on the list of criteria as to what truly makes a course great (which I think are the most important criteria).

[Patrick_Mucci]

Some people have a knack, or talent for evaluating a course architectually.

Many of them aren't on any panels.

Some people don't have that talent.  
Unfortunately, some are panelists.

Perhaps the criteria and the panelists need to be re-examined.

[Mike @ Kiawah]

I know that one of the GD criteria for becoming a panelist is the candidate must be a 5-handicap or better.  Like what has been mentioned here, I know of some bogie golfers who have a keen eye for architecture and some scratch or better who wouldn't know a great design if it jumped up and dumped dirt on their shoes...

[Ron_Whitten]

So the GD rankings (double entrendre, as Tom Doak used to say) have become the latest whipping boy for the GCA crowd.

You're missing the point if you can't understand how Bethpage Black falls off the list during a remodeling by Rees Jones. We want at least 30 evaluations of a course in order for it to qualify. (Nothing objectionable about that, is there?) You can't get 30 people to play a course while it's torn up, and, given our deadlines, wecouldn't get enough of them to it once the course reopened. So it dropped off the list for a time. Was it a mistake? Yeah, not to prod enough volunteer panelists to get over to it. But was Rees Jones a guarantee that Bethpage would still be a 100 Greatest Course? Of course not.

The 100 Greatest isn't at all like a list of 100 Greatest movies of all time. There is no "all time" involved in this survey. The problem some of you have with our 100 Greatest is they expect a Hall of Fame, but it's merely a magazine list that changes every two years, and should, because tastes in architecture change that rapidly.

For all the groaning about Tradition, I'll remind you that it helps balance the hype over new courses that otherwise cost older courses positions on the list. Which do you want - Pine Valley at No 1 and Shadow Creek at No. 8 or Pine Valley at No. 2 and Shadow Creek at 31st? (I know you guys. You want Shadow Creek off the list.)

How can a course go from 100 Greatest status to also-ran? Gosh, played Wild Dunes lately? It was once a terrific natural dunes course. After being ravaged by a hurricane and condo builders, it's a shell of its former self.

Seen the housing along Troon North lately? Or the condition of Oak Tree until this year? Had we continued to list those courses, you'd guys would be complaining that we're celebrating yesterdays stars.

Is it a perfect system? Of course not. We can't get the same people to all courses. Is it the only magazine list that rips back the curtain and shows its entire process, warts and all. Yeah, it is. Which provides writers like Geoff Shackelford and Gary Galyean, and publicists like Mike Vegis, grist for a comedy column or sanctimonious sermon or stern letter of indignancy. Free country. Fire away.

Meanwhile, realize that the secret to Golf Digest's 100 Greatest is that it's sucked many of you in. Love it or hate it (and most of you that post on this site obviously hate it), you're still reading it. That's what a magazine article is supposed to do.

The day a 100 Greatest list comes out with absolutely no changes is the day magazine sales plummet. And also the day some people quit taking an interest in golf course design. And the day I probably get out of the course ranking survey business once and for all.

Thanks for listening. And keep buying that magazine.

[Scott_Burroughs]

The reason for Bethpage falling out of the rankings for a 2-year period, not enough evaluations, is the same reason Fishers Island was never on the list for many years, not enough people could get to it/on it.

But the reasoning that new housing bordering a course somehow lowers a ranking is bunk.  It doesn't detract at all from the strategy of the holes, provided they're far enough away.  I hate homes on a course, too (#1 Pebble has quite a few reaching out and touching the course). I guess it must lower the "Aesthetics" ranking in GD.  And conditioning?  Bethpage barely scraped on the list a few years ago before it had the now-tournament-like conditions.  Tournament conditions does not a great course make.  Let's not get started about Rees Jones.

[Gib_Papazian]

Geoff,
Please take this in the spirit it is intended, because you know how much I enjoy reading your work.

That said, I have trouble with the tone of the piece, not because you do not make excellent points (as usual), but because it comes off as a terribly condescending and offensive condemnation of a pretty decent rating panel.

I know several GD raters very well and although their opinions often differ greatly from mine, they can articulate and defend their positions just as cogently as I can my own pearls of wisdom.  ;-)

You do not like the fact that GD, as a group, tends to reflect the changes of fad and fashion in American golfers - but I think that is going to be endemic anytime you assemble a diverse panel of 800 people from every corner of America.

For example, panelist tastes have drastically changed in America since Atlantic opened 10 years ago. Most raters I know - regardless of which panel - think it's just awful. But it was "Best New" in 1992

Do I think we (GolfWeek panelists) are better versed in the finer points of architecture? Of course we are Geoff, but there are only 180 or so GW panelists. We cannot possibly cover the necessary ground to stay as current as GD - even if our  opinions (as a group average, not individually) are less affected by new and glitzy tracks.

Plus, because our GW lists are segmented, we do not have to try to compare Merion with Galloway National. Trying to compare a links and parkland course using the same scale is tough enough, but two eras also?

I'd rather do it our way.      

And believe me, as a group we tend to have our own biases. The only difference is that our dogma is slightly different - but it still reflects the ideas of our leader.

I'll bet if there was some way of determining who the best (read: well versed and traveled) 50 were from GD, GW & GM and brought them en masse out to a new course, the evaluations would be nearly identical.

Personally, I think you have to be able to play the game reasonably well to rate a course properly, but that includes the many 18 handicappers I know who seem to grasp architecture beautifully.

I also think you can (rare disagreement with Dave Wigler) rate a course just fine by looking at it.

That said, it requires a lot of imagination to pick out and visualize the strategic rationale for the little rolls, folds and swales that make the difference between good and great.

I like to think I was able to grasp everything at Westhampton pretty well just driving around with Mike Rewinski and Neal. Perhaps it is more difficult later to rattle off the sequence of holes because the frame of reference is different.

RJ Daley makes a good point about making an unqualified evaluation in one pass, but in my case, I find that writing a column about it later tends to confirm or change my intial reaction.

Assuming that the piece is not just empty hyperbole, penning 1200 words on something forces me to fully understand not only what I've seen, but also helps sort out the inevitable conflicting impressions you get when playing or looking at a golf course.  

[Patrick_Mucci]

Ron Whitten,

I had difficulty understanding your statement that the magazine changes the ratings every two (2) years because tastes in architecture change that rapidly.

That would seem to imply that there is no stability, no constant, no standards that continue to pass the test of time, only fads when it comes to the evaluation of golf course architecture, and I'm not sure I would agree with that.

You're right about controversy and debate selling more magazines, which I guess is the ultimate goal of the publisher, but doesn't the fact that a new course ranked so high in the ratings, only to disappear shortly thereafter mean that the initial raters got it wrong in the first place.  Aren't there enduring values in architecture that should last beyond two (2) years ?

I was also troubled by the housing issue.
There are plenty of houses and hotel buildings at Pebble Beach.  If Pine Valley, NGLA or other top courses suddenly had houses on their perimeter, would that detract noticeably from the architectual values of the golf course ?  Values that have existed and have been apprreciated for decades ??

I wasn't bashing raters at your magazine or any other magazine.  I was only making a point that the panelists at different magazines have differing abilities, as do non-panelists.  I do feel the criteria and the strength of panel need to be re-examined periodically, with respect to their credentials, or rating abilities.

Like a wedding, birthday, or anniversary, there are only so many invitations to the party, and many want to champion their selections or regional favorites, so it's not an easy task to distill all the information and potential bias to reach an ironclad conclusion or perfect list, and I don't think many people expect that of you.
But, they do expect a reasonable consistency
I think that may be the chief complaint.

While no one suggested that the ratings be perfect, it doesn't mean that the raters or the process can't be re-examined for better efficiency.

Eternal vigilance is the price of greatness, and without constructive criticism, progress is impossible.

[Tommy_Naccarato]

David Wigler and Spoon,
What I'm trying to ask with my statement is exactly this, "Are you into golf architecture and golf courses enough to where it is a passion to be able to go see them without playing them."

Take me for instance:

I have yet to play The Valley Club At Montecito, but I have seen the course several times. The bunkering tie-ins just amaze me, which add to the deception, which I think is just one of the many things that add up for a course to be considered viable for ranking. There are also many other strongpoints to the course. Some of these I have even seen in many modern designs, and I beam with joy! Something did get through from the grand masters! If I never got a chance the rest of my life to play The Valley Club, I would still know that the Valley Club is a very special golf course that deserves the undue praise it laments. It isn't the best MacKenzie-designed course in the world, but it may rank as one of the most special. (In fact, I think they all do!) I equate this to the charm the course exudes, from it natural surroundings and the bit of quirk that still resides there.

I love quirk!

I think TE Paul, a very good golfer can elaborate on the benefits of looking at a golf course without playing it. There are times I feel that I'm better off, not letting my flat rag tag and chubby swing get in the way. Surprizingly, I have found out on many occasions that by not playing, I have seen things I did see before, if it is a course that I have played before.

Some examples of me playing horribly where the course scored highly:

Pine Valley
Merion
Plainfield
Riviera (Played from the blue tees after five straight days of golf)
LACC
SFGC (In a quagmire of rain, where I caouldn't even hold on to a club!)
Astoria G&CC
And many, many more.

I think the bottom line is that maybe SOME panelists are maybe not as passonate about the architecture as I am. Maybe I might be a little hard, but in truth, I want to inspire them to learn what great golf design is all about and maybe they could teach me something I don't know!

[DIYDo DIYDon't]

You know the old adage about not being able to please "all the people all of the time" well it is never more prevalent than when people start taking pot shots at panels and lists on this DG.

I have personally heard some people complaining about how a particular course in my state can stay on the GD 100 Greatest list even though it is currently closed and undergoing renovations.  They were critical about "how that just proves how out of touch this list is when a course isnt even open and it is still listed in the top 100!!"  Now I read above of an instance where a course closed for major renovations (Bethpage) fell off the list and was not listed as it was reevaluated.   And SURPRISE SURPRISE people are critical of the list for of all things NOT KEEPING IT ON THE LIST.  

You talk about Damned If You Do, Damned If You Don't!!!!

And anyone who can sit here and with a straight face say that Condos/Houses now lining both sides of fairways that once were pristine forest on a new course does not change the playing experience of a golf course needs to have a special padded golf architecture library built for them.  Because while it may not change the "Pure Strategic Elements" of a particular hole it certianly does change the feel of the hole. And there is more to a hole/coures than just purely strategic considerations. Sometimes architectural snobbs get so high and mighty about the sterile strategic elements of a hole that they forget one of the most important aspects of what a golf is.

IT IS A GAME!!!!
PLAYED BY HUMAN BEINGS!!!!
(and contrary to many peoples beliefs)
IT IS DONE FOR PLEASURE AND ENJOYMENT!!!

When you try to put sterile strategic interpretations of a golf hole and say that the surroundings have absolutely no effect on a hole you have removed the human emotion we call pleasure.  And the day I do that is the day I quit the game!!!

While I enjoy the strategic elements of a great course as much as most anyone.  Call me weak, but I also find beauty in many outside elements that envelope the induvidual holes.

ps  for those of you that still hold on to that black and white view of things I would submit this also.  I have seen courses where the recent addition of condos in place of the trees that were there the first time I played the course have changed the strategy.
1 - because now there is a monument to white PVC pipe lining the inside edge of a dogleg that changes the risk/reward ratio of playing closer down the inside edge of the dogleg.  
2 - Also that great towering pine tree I once used as a target for my tee shot has been replaced by a Primestar Dish on top of the monstrous new home that stands were the pine tree did, or the bushes you used to use for a referance have been replaced by some lovely wicker chairs on someones deck.  But I guess I am just a little softie and for that I apologize.