Why GOLF DIGEST is wrong to use Tradition/100 Best

[John_Conley]

Golf Digest continues to call their list the “100 Greatest” while intentionally altering the result of their raters with an extremely questionable “Bonus” category called “Tradition”.  In school, students are sometimes given the opportunity to earn Extra Credit.  If Golf Digest were the teacher, the Extra Credit project would be worth as much as the midterm.  

Tom Huckaby, a GD panelist whom I consider a friend, continues to defend their use of this category.  In doing so, he is also making the comment that he and people like him alone are not capable of generating a good outcome from their scoring efforts.  Because of his steadfast resolve, I keep wondering if I am the one who doesn’t get it.  I’ve listened for four years with an open mind, and I have yet to hear any justification that makes sense to me as to why the Tradition category is given so much weight.

A decision to give so many points to the category is one thing, but then the subsequent misapplication of two-pronged criteria becomes equally troublesome.  They state that Architectural and Tournament history determine one’s score, when study shows it hardly is that simple.  Name recognition, which is determined as much by age and appearances on previous “Best” lists, carries as much or more weight than tournament pedigree.

Some examples:

The Prince Course has more Tradition than The Ocean Course, at least according to GD.  Keep in mind that TOC has hosted a Ryder Cup and a World Cup.  Both are 12-13 years old.  Is RTJ, Jr’s Prince more architecturally significant than a course that proved a true master could work in haste to brute-force a great course under heavy pressure to host a high-profile event?  And don’t say it isn’t a great course, the GD data indicates that it is.

Sand Hills registers one of the lowest scores of all Top 100 courses in this category.  If Architectural history means anything, it sure would seem that a course eschewing modern construction standards because it sits on the equivalent of golf’s Holy Land should get points for being worked on by someone brilliant enough to realize this.  Ask any architect and they’ll tell you the work by C&C was less intrusive than anything they would have envisioned – and turned out wonderful.

Interlachen and Hazeltine are the two Minnesota courses that stay on the list.  The former has hosted events won by Glenn Collett Vare and Bobby Jones when he won the Impregnable Quadrilateral and boasts Patty Berg as a former member.  The latter is now in bed with the PGA of America and will host another PGA Championship and a Ryder Cup in the next 20 years.  Add that to their four U.S. Opens (two men’s – David Graham and Payne Stewart over Scott Simpson in a playoff, one women’s – one of Hollis Stacy’s THREE, and a senior) and the fact USGA President Reed Mackenzie is a member; it seems like only a matter of time before it passes Interlachen.  Or is it?  That club has hosted the 1986 Senior Amateur, 1993 Walker Cup, and 2002 Solheim Cup and has a future U.S. Women’s Open on the calendar.  These events, smaller in prestige, are possible while tournaments that host a bigger crowd are not.  Should this matter if we are talking about the golf course?

More to come…

(Strip out the Tradition and the GD list looks great.  Or at least score from 0-2 instead of 0-10.)

[THuckaby2]

Great stuff, JC.  

You did, howver, HUGELY oversimplify my defense of this system.  I have no energy to restate it again, but it goes way beyond simply saying "he and people like him alone are not capable of generating a good outcome from their scoring efforts."  I don't believe I ever said that, not so plainly, not so simply.  What I did say is in many other posts on this subject.... and it boils down to I believe those with TIME to devote to the study of tournament history and architectural significance are better qualified than the vast majority of "raters" - that's why I feel the addition is justifed.

I'm sure you'll find many results were it came out "wrong" by your thinking.  Hell I'll agree with you on most of them, I'm sure.

But it's not gonna change my thinking on the process.

You do make powerful arguments though, and I'm sure 95% of the regulars here agree with you.  Such is life....

TH

[THuckaby2]

Damn - I must have missed that part of the deal.   I gotta look more closely next time I rate a course.  

TH

[Andy]

John, like you, I share some real concerns with how the process works.  As a matter of fact, it is not a stretch to say the Golfweek and Golf Digest have a "big brother" manipulator prior to publication.  I do think there is some rationale for this as clearly among the raters on both teams, there are those that have odd scores that might or might not need to be thrown out.  I personally like the Golf Magazine system, less people, more people of repute and power who cannot be manipulated by free golf or an exspensive dinner.  Granted, the GM system is less "precise", but I do like the fact that there is not some ridiculous method of "control" that allows what you mentioned at The Prince and Sand Hills.  I have just thought, generally, that their rankings are more accurate and they also don't have the age requirement stuff, which I find quite odd.  Anyway, just my thoughts.  

[THuckaby2]

Andy:  I'd agree that Golf Magazine comes the closest to getting this "right", if such a subjective thing can ever have a correct result...  the use of a small knowledgeable panel not easily awed or swayed is the best way to do this without a doubt.

I do however believe that their "I know it when I see it" methodology just takes it TOO FAR to the extreme of subjectivity... their panel also doesn't have a very good representation of the "common man" and what thrills him in a golf course, and given that's the vast majority of golfers, that take needs to be represented....

So none of the magazines does this perfectly.  Each measures different things, in different ways, and each does a fine job.

Brad Klein remains correct in any case - these rankings are never going to make anyone happy.

TH

[John_Conley]

Tom:

I am oversimplifying because the final Top 100 reflects the result of the decision to include Tradition and weight it as they do.  Make it a max of 2 points and it wouldn't negate the efforts of the panel.  THERE ARE NOT 8 point swings in any other category for the population of courses we are looking at.  (Every other category has nearly every course scoring between 7.0 and 8.9.  Doesn't this effectively QUADRUPLE the weight given to the Tradition factor?!  {It does, I'm pretty strong statistically and can't believe the powers that be at GD either don't see this or want a flawed result.})

Can a course appeal their score?  They should be able to in this category.  If each club presented a resume it is possible some meritous events occurred in the past of which the GD arbiter was not aware.  And don't say the past doesn't matter.  If it doesn't, a course like Garden City would only have Architecture (not Tournament) points.  (Can't really count their own club event, can you?  Or do you?  Augusta is getting credit for their's.)

I actually have defended the work from Golf Digest panelists.  It just pollutes the list to destroy the data with an erroneous category given an unusually large weighting.  Why isn't it worth the same as walking?  You defend it, which implies it makes sense to you.  Shouldn't it be lessened in significance?  It is okay to agree.  I often disagree with a group decision when I am a part of the group.

[THuckaby2]

OK, you've won me over JC.  There's enough evidence to say the process could be done a lot better - obviously.

It just is what it is, and to me what it is isn't all that bad (that was one hell of a sentence).  I can live with it.

Fight the fight, I'm out.  Just don't blame the whole thing on me!

TH

[John_Conley]

Andy:  I'd agree that Golf Magazine comes the closest to getting this "right", if such a subjective thing can ever have a correct result...  the use of a small knowledgeable panel not easily awed or swayed is the best way to do this without a doubt.

That's funny.  

A golf course architect I played with told me he didn't really put much stock in that group BECAUSE of the celebrity/accomplished player aspect of their team.  With so few voters, how long will it take before people see the Wild Horses, Links at Red Mikes, and Quarry Ridges?  

Tom Doak realized this and came up with the "Hidden Gems" addition for courses like Cruden Bay that scored highly and weren't yet seen by enough raters.  No surprise that courses appearing here often "graduated".

No methodology is flawless.  But to compliment guys "who won't be swayed by a free steak" is way off-base when the assignment necessitates seeing A LOT of new courses to stay current.  Did Shark's Tooth make the GOLF Magazine Top 100?  How about Jim Engh's Redlands Mesa in Grand Junction?  I'm not saying they should or shouldn't, but they won't when you are relying on guys like Bryant Gumbel and whoever they have that still plays golf for a living to go see it.  Also, don't they still include architects?  While we all have biases, their's are too big to set aside.

Golf Digest - Extensive national team ensures "coverage" and result is very good until the "hand of God" massages the numbers.  Too hard for new courses to make the list even when deserving.  With all the new courses in the last 20 years, an increase to Top 150 would be just as meritous as Top 100 was in 1978.  Difficulty still counts for more here than on other lists.

GOLF - Interesting because it relates U.S. and foreign on one list.  Top 100 is essentially the same result as Golf Digest and GOLFWEEK, which indicates many similarities no matter the process - these are truly America's best courses.  People comment on the exceptions, of which their are few, indicating that the results are mostly similar.

GOLFWEEK - Actually Top 200 with the 1960 distinction.  Take the first 60 Classic and 40 Modern and everyone would be happy.  Most ire from observers contained to discontent with the Bottom 50 Modern, where everything blurs - right in with the 50 or so that just missed the list.  Point is that there may be fewer "great" recent courses.  Inclusion of the extra 40 Classic rewards many great courses the other lists can't; 100 Modern means notable new courses can attain a ranking - nearly impossible from the other two lists.

Special note:  GD "Best in State" - Not presented in the same format, but very similar in end result as the Top 100s from GW.  It is more inclusive.  No matter who's counting, a swing of several places is easy here because the differences are much smaller.  (logarithmic vs. linear, for you engineers)

[Andy]

Tom, I do think a hybrid of all, the smaller group that GM has, along with their lack of "age" rules, take some of the precision and detail of GD(but with tradition weighted differently), and then maybe some of the "bold strokes" of Golfweek, and you might have a Top 100 that at least a majority of people here might go for.  However, that would be no fun!  I totally agree, no one group has it right, and no one group is all wrong, and, just like golf shots, someone is happy and unhappy with each ranking we see.

[THuckaby2]

JC:

This would be so much easier in person.

I didn't mean GOLF MAGAZINE'S panel composition is perfect, though I see how my post could be read that way.

What I meant was the idea of a small, knowledgeable, devoted panel above being awed is the best way to do this.  Obviously coverage would become the issue there, but given this is all hypothetical anyway, let's assume that's all they do and they really get around.

GD's panel has some great people on it, but it too is obviously far from perfect.

PLEASE stop quoting me!  I'm in enough trouble as it is.  

TH

[John_Conley]

Tom:

Three bodies I know of use "ordinal" scoring.  Figure Skating, my Fantasy Basketball League, and the Fitness America Pageant on ESPN2 (half routine, half swimsuit).  If you used ordinals for all of these categories it would mute the influence an 8 point differential has on Tradition, where you can see I still don't understand the implementation.

I think it is a bogus category, but I can live with it if it doesn't have such a huge impact on the results.  (No.  I didn't misspell it.  I meant boGus.)

If "Shot Values" are doubled, why is "Tradition" QUADRUPLED?

[THuckaby2]

JC - please direct this to the GD editors.  I remain happy with how they do things, just allowing that it is imperfect but it's fine for what it does.  I'm neither the statistical marvel you are, nor do I care about this as much.
 
TH

[John_Conley]

JC:
PLEASE stop quoting me!  I'm in enough trouble as it is.  

Hucks:  

I have used quotations from your comments and I have taken the liberty to PARAPHRASE you as well.  You can't get in trouble for things I say.

I agree in part with you regarding GOLF, but I also felt it necessary to offer the other view.  I think all methods will be imprecise to some degree.  That said, I challenge anyone to show that Golf Digest and Golf Magazine have done as much to wring out the part that can be improved as much as GOLFWEEK has - yet "our" lists get as much scorn as anyone.

Golf Digest has it about as good as it can get.  And then they mess it up with their inappropriate application of "Tradition".

[THuckaby2]

Hey John, Golf Week's list never got anything but praise from me.  Oh, I did enjoy when those guys were on the hot seat, but only because I know a bunch of them and shit-giving is our s.o.p.  My take is that GW takes the middle road on this - trying to be subjective and obective - and they do a damn fine job.  Maybe that list is the "best", although I think they wuss out by separating it into Modern and Classical!  

In any case, many ways to do this, lots of good, a little bad.

TH

[John_Conley]

JC - please direct this to the GD editors.

Who runs the list?  Tell him to call me if he/she wants.

I e-mailed GOLF through their website once about the idiocy that was PAR 2000, or the greatest 500 holes in the world.  What it was... the 500 best holes on courses that are already considered great.  It was a stupid gimmick designed to steal interest with Y2K nearing.  Anyway, I never got a response.  (My gripe: There was absolutely no mechanism to include a great hole in Redwood Falls, MN or Truth or Consequences, NM and then to profess you were listing the 500 Best was incredibly pompous.)

Pacific Dunes and Sand Hills can beat a course across the board and be ranked lower when The Man chimes in with Tradition.

Someone points something out to Brad and it at least is taken under consideration.  Is that the case with GD?  I'd think long-time panelists would be saying, "Quit messing up the great work we do."

[THuckaby2]

JC - it's not gonna be me that sics you on the GD editors in reality - as I say, I'm likely in way too much trouble as it is.  

My point was I have grown tired of being the sole defender of the GD list.  I keep trying to give up, but you keep directing more back at me... Not that I have any need to have the last word - far from it - I just don't like to leave questions unanswered, that to me is impolite.

Can we just leave the bottom line as each magazine does this decently for their own purposes?  Or is even that not good enough for you when it comes to GD?

Dave Wigler's thread asking for my castration seems to me to be the best take on this, given how it underscores the absurdity of the importance people place on these....

Big sigh.  Back to Brad's article on the subject...  

TH

[John_Conley]

My point was I have grown tired of being the sole defender of the GD list.  

Tom:

I wouldn't try to defend something I didn't agree with.  It doesn't look like you are in favor of quadruple-rating Tradition either.  Yet you tried to convince me it was a proper category!  You say you "gave up" when you realized you and I agree on much of this matter.  (Pretty sure we always have.)  

I haven't been arguing with you, so please don't get that impression.  If anyone could explain the methodology in a way that made sense to me I would be happy.  No one has.

I started the thread to voice an opinion.  I welcome all thoughts as to why I'm misguided.  It seems that people on this board don't agree with the weighting of Tradition either.  I always say, "if you don't agree with the process, don't complain about the result."  I have not complained about anything on the Top 100 and won't.  If you panelists used a process I agree with - and stripping Tradition or lessening its impact is all it would take - the only gripes from anyone would be petty.

Keep up the good work.  If they ask for feedback on how to improve, please share my comments and label them Idiot (which I was called two days ago) or Outsider.  If they don't ask, it sends the signal that they don't care what you 800 think.

NO OTHER MAGAZINE HAS VASCILLATED AS MUCH ON HOW THE RANKING IS COMPILED, even if you strip out the beginning for them being the innovator.

[THuckaby2]

JC - I've obviously never looked into the methodology as closely as you have, so I defer to you.

But I'm still gonna say although it is imperfect, it's good enough for me.  I'm just not into nit-picking every last detail, and again, refer back to my many other ways I've tried to say very nicely that people take all this way too seriously. This issue of "bonus tradition points" remains fine by me, because for me it's a decent attempt at trying to include things I feel need to be included.  If the exact details of how they do it are imperfect, than what the hell - to me its better than not doing it at all.

As for vascillating on how they do things, that too doesn't bug me - changing things to try to do it better is always going to be ok by me also.  Maybe GD succeeds, maybe it fails...

TH

[DMoriarty]

Has anyone compared 2003 "Tradition" numbers to past "Tradition" numbers?  I looked online but couldnt find any past data.

If the "tradition" points substantially vary from year to year for particular courses, it would seem that "tradition" is purely being used as a fudge factor.  

[THuckaby2]

That's a damn good point, Dave.  Hard to see how any of those factors combined to produce bonus tradition points change much over time... I guess a place like Bethpage ought to get more now having hosted the US Open, but that'sa unique case and the vast majority of courses don't change much in ambiance, tournament history or architectural significance from year to year.

Of course even if these numbers did change, it could as much be different people assessing such - with different opinions - as it could be a fudge factor.  And remember with ambiance counting for 40%, and the hugely subjective nature of that definition, that could account for change also.

So it's a good point, just not completely cut and dried.

TH

[John_Conley]

Has anyone compared 2003 "Tradition" numbers to past "Tradition" numbers?  I looked online but couldnt find any past data.

If the "tradition" points substantially vary from year to year for particular courses, it would seem that "tradition" is purely being used as a fudge factor.  

D:

I was a step ahead of you, but lacked the sheet.  I thought I had it.  I'll look again.

[John_Conley]

Found it!  Now let me analyze.

[Carlyle Rood]

Tradition should be an integral part of the course rankings.  It is a fundamental and legitimate component of a course's character and enjoyment.  And I'm certainly not confining a course's tradition to the tournaments it has hosted or the architects that designed it.

For example, I think of the 13th hole of my home course of Atlanta Country Club.  It's a dramatic par 3 over a waterfall that runs into Sope Creek.  It's both scenic and exciting to play.

However, another important element of the hole is its proximity to the ruins of a Confederate paper mill along Sope Creek.  From the tee, you can look down the steep embankment along Sope Creek and see the ruins.  It's easy to find yourself daydreaming about the Civil War (or War of Northern Aggression as I prefer to call it) and you're immediately transported back in time 150 years.  It's a fundamental element of the experience on that hole.

You can design comparable holes with equal shot values.  You could even create a replica of the hole.  But, you'll never be able to reproduce the experience of the hole without the paper mill.

Carlyle

[THuckaby2]

Agreed completely, Carlyle.  My example when this came up before was Pebble Beach #17.  Like the hole or hate it (and it surely isn't universally beloved), the fact that Jack hit the stick with a 1-iron in 72 and Watson holed out there in 82 MATTERS, at least for us old enough to remember such things!

So the fact those things happened - which would come under Tournament History the way GD does it - does matter... because a lot of people play the hole with that in mind.

Of course there are many weaknesses with how it's done by GD,and John and others have pointed this out.  So no need to point this out again John if you read this!  The point here is that Carlyle is right that these intangible things do matter... and if that makes it tough on new courses, well... tough shit!  They can make up for such in other ways, or just be patient.  The answer surely isn't to discount this at courses where it does occur, and is a positive....

TH

[John_Conley]

.............2001.................2003

HI...........8.39.................8.55 - both Oakmont  (why, I have no idea)
LO..........0.04.................0.42 - Sand Ridge & Pac Dunes

Sand Ridge and Crosswater fall off from 2001, precisely because they lack Tradition.  With an average score here, I'm sure they'd be on the list.  (P.S.  Sand Ridge was previously #55.)

Sand Hills...0.62.................1.15
TPC..........3.33..................3.80
Valhalla.....1.49..................1.97
Point O'Woods 4.15.............4.82
Crooked Stick 4.13.............4.24

Leads one to think that these are just points you accrue over time.  Why would an older course go up more than a newer?  (PO'W vs. CS)

However, look at this!

Augusta....7.56...................8.11
 
as the course was changed even futher from its Architectural heritage. (I know, it continually evolves and always has.  Wouldn't that preclude it from much on the Arch side and negate some of its Tournament kudos?)

SFGC....5.66......................5.56

How the heck do you go DOWN in Tradition?

Crystal Downs 3.74.............4.20

Another move like this and it will someday have as much tradition as Crooked Stick!

GdnCty ..5.82.................6.50

Has the measurement changed in the last two years, or has that course (actually it would be the club that has a hand in this category, not really the course) done something substantial to increase their tradition?  Are the finally getting credit for having Pat as a member?

Hazeltine  4.39.................5.27
Interlachen 5.90...............6.47

Both significant moves helped by a PGA Championship and Solheim Cup.

Homestead 5.90...............6.37

Riddle me this... how can that even be mentioned in the same breath with Interlachen?  Is the U.S. Mid-Am that big of deal?

Aronimink 4.93.................5.54

The difference between #51 and #60 is just a 1/2 point.  Take .5 away from one course and give it to another and you have an effective separation of 1 point - or the difference between #75 and #94.

If Oakmont is top dog because it ushered in Penal architecture, please explain...

Harbor Town  5.16.............5.03

which also goes down and should be rewarded with more pionts.  Or did Crooked Stick take theirs?

I'm done editorializing.  Draw your own conclusion.