What Changes Would You Make to the Top 100 Rankings Criteria?

[Sam Kestin]

Each of the various publications has their own categories in which they give courses grades--these categories are then weighted in various to ways to produce a score for each course. The courses are then ranked according to that overall score.


The current categories for Golf Digest are Shot Values, Resistance to Scoring, Design Variety, Memorability, Aesthetics, Conditioning, and Ambiance. Every category counts equally save for Shot Values--which is doubled before tallying the total score.


If you could design the Top 100 Courses rankings criteria for Golf Digest, what would your categories be?


What would your formula be for turning the categories into a final number?

[Ulrich Mayring]

I do it this way:

https://sites.google.com/site/golfcurriculumvitae/gcescale

Ulrich

[Sam Kestin]

Ulrich--


Looks great and really interesting. I like the concept of "Flow"--I think a highly under-appreciated aspect of golf course design is the batting order of the holes. Imagine Augusta National in its original incarnation--with Amen Corner coming as the 2nd-4th holes. Would be a completely different tournament with a completely different feel...all because of a change in the flow.

[Sean_A]

I wouldn't have any criteria and instead would rely on the experience and knowledge of raters. I highly suspect that often times ratings are "fixed" to agree with the overall rating any given rater may think/feel about a course.


Ciao

[Ulrich Mayring]

Another thought:

As you can see, I folded "conditioning" into the "shot values" criterion. Many systems have conditioning as a seperate aspect. But I wonder what does conditioning in and of itself actually do for me? Does it add to the enjoyment of a round and if yes, why?

I think for the most part all it does is give me better lies and thus make the game easier. I make more putts on true greens and hit better shots off a consistent fairway. So that would basically contradict the infamous "resistance to scoring" category. But in ranking courses I would try to avoid contradicting criteria.

There is one exception, where conditioning actually enhances the course and that is when there is a ground game. You need firm and fast for that. But many courses are designed for target golf and thus firm and fast does nothing for them, it may even be detrimental. So conditioning in my mind is not a very good criterion for ranking arbitrary courses. For the most part it just makes the game easier or, dare I say it, fairer.

So I decided conditioning does play a role, but does not sit in the front row.

Ulrich

[Jud_T]

Trying to assign mathematical rigor to a qualitative artistic critique gives the false impression of statistical significance.  This is why I don't watch Figure Skating or Gymnastics in the Olympics.  Better to rely on the opinion of one person who you trust than try to get a consensus by imposing categories and tallying "data"...

[Peter Pallotta]

I find on many of the modest public courses I play the same quality/characteristic, i.e. in trying to ensure they are "playable for all" they end up being "engaging for none".

These modest/average courses (with modest aspirations gca-wise and modest budgets, and average amenities and an average clientele) provide width a plenty, and hazards too (of a sort) and multiple tees and contoured greens -- but there is almost no (golfing) rhyme or reason to any of it; the relationship between teeing grounds and landing zones and fairway hazards and green shapes/contours rarely punish anyone but they don't reward anyone either.

It must be quite a challenge for an architect to design a course that is indeed playable for all but that is also genuinely engaging for all (as I found that some of the higher end and, for a while, well regarded publics I've played also seem to share this same characteristic).

I think an architect who does manage to achieve this feat has what I'd called "Golfing Sense", and a course that makes this characteristic manifest has a "Golfing Sensibility".  And the better an architect/course manages to actually be playable while also being genuinely engaging to a wide range of golfers, the higher the score in these two areas.   

So, in short, I'd add a category worth 20 points out of 100, i.e. "Sense and Sensibility". 

[David_Elvins]

20%  - Percentage of course grassed with Fescue (or fescue/bent/native blend)
20% -  depth of pitch marks on greens.
30% - width of fairways.
30% - variety of holes lengths.

[William_G]

needs a fun factor
memorable?
walkability

[Ulrich Mayring]

Trying to assign mathematical rigor to a qualitative artistic critique gives the false impression of statistical significance.

Well, any intelligent writer needs an intelligent reader. The real question here is whether golf courses can be compared at all. If you deny that, because you think golf courses are works of art, then any attempt at ranking them is ridiculous. There is no ranking of old master paintings out there that pits Rembrandt against Leonardo da Vinci.

If, however, you believe that golf courses do possess artistic qualities, but are largely the result of craftsmanship, then they can be compared along certain criteria. Any craft product, even something as taste-dependent as wine, can be ranked that way.

I am of the latter opinion and try to fold the artistic qualities (which are 100% subjective) into each of my categories. But I also think golf courses are very similar (18 holes, all have fairways, tees and green complexes, most have bunkers or water hazards, all have a variation in length, shaping or soil) and, more importantly, those similarities have a bearing on how much fun I have playing these courses.

It's different with old master paintings, they also have common properties (amount of yellow paint used, panel material, age), but those properties have little bearing on how much I enjoy the painting.

The advantage of having a certain number of categories (don't overdo it, obviously) is that my ranking gains independence from subjective circumstances. For example my mood or my score on the day I played the course or even the agreeability of the folks I play with. If I simply assign a single number for "how much I enjoyed my round", then any course I play in bad weather is at a natural disadvantage.

That's a bit like saying I didn't enjoy the Mona Lisa, because the crowds were so huge that day and I had little time to study it, so for me it's not a great painting.

Ulrich

[Thomas Dai]

Some suggestions for criteria -
Amount of water used on the greens, fairways and rough.
Number of animals grazing the property.
Number of buggies available to hire by the non-disabled.
Number of rental club sets available for hire.
Length from previous green to next tee.
 
Atb

[Jud_T]

Ulrich,


You certainly can rank courses by any criteria you like.  I may or may not agree with your criteria and their respective weights.  The issue is when you take your criteria and get 100 guys to fill out a form based on criteria and get a consensus.  Does everyone really view scenery and shot values the way you do?  As Sean says, are they just fitting the numbers to their personal preferences?  What impact does the consensus view have on the exercise?  Then I, the reader, come to your list with my own set of likes and dislikes.  I have to try and retrofit my desires with the list using only summative numerical criteria instead of a qualitative written review where I can get a better sense of whether I personally may like the course and want to seek it out.  Furthermore, the averaging nature of a group-think exercise will by definition hurt unusual courses and reinforce the status-quo.

[John Kavanaugh]

I only wish raters could stay anonymous.  Me and a couple of civilians were recently paired with a guy that obviously was a rater. I've got to give him credit, it only took him 7 holes to specifically name drop his magazine and mentor. I have never been so in love with nature as in that moment when the crickets sang a chorus of who gives a shit in unison with the sound of our cleats brushing against the fescue as we rushed away from another barrage of misinformed regurgitated pablum. Just a tad of advice, if you are going to drop your rater card, have something fresh and interesting to say, or don't say anything at all.

[Tom_Doak]

When I get home I am going to dig out the names of the panelists for GOLF Magazine the first time I helped with their list.  It was quite different than today.  I agree with Jud's points about trying to make a subjective exercise look objective by introducing lots of numbers, but the most important rule of data processing is still GIGO - garbage in, garbage out.

[Sean_A]

Ulrich,


You certainly can rank courses by any criteria you like.  I may or may not agree with your criteria and their respective weights.  The issue is when you take your criteria and get 100 guys to fill out a form based on criteria and get a consensus.  Does everyone really view scenery and shot values the way you do?  As Sean says, are they just fitting the numbers to their personal preferences?  What impact does the consensus view have on the exercise?  Then I, the reader, come to your list with my own set of likes and dislikes.  I have to try and retrofit my desires with the list using only summative numerical criteria instead of a qualitative written review where I can get a better sense of whether I personally may like the course and want to seek it out.  Furthermore, the averaging nature of a group-think exercise will by definition hurt unusual courses and reinforce the status-quo.

Jud


A good example of just letting raters do their thing is the Unofficial GCA.com list.  It is the best list I know and there weren't any categories.  As you say, the category method really lends itself to a false sense of science/math when this isn't the case at all because the categories, weight of categories and opinions expressed by raters are 100% subjective. 


Ciao

[Peter Pallotta]

Sean, Jud, Tom -

As you know, I am not a rater and don't tend to rank courses, and many of the "categories" used in the GD ratings process have no great meaning/significance to me.

That said, the approach you are all espousing doesn't seem all that satisfying either. I understand your belief that such ratings/rankings are all "subjective" -- but if that is the case, then let's cut to the bottom line and once and for all agree that all we can say (and title any given list) is:

"On this day, in the year of our Lord two thousand and sixteen, a solitary golfer named Sean/Jud/Tom -- fully recognizing the inherent subjectivity of the entire endeavour; the reality of his own subtle or overt biases regarding golf course architecture; the inherent and inevitable limitations of talent, temperament and financial resources; the fluid nature of human memory and personal narratives and hierarchy of values; and the undeniable influence that weather, turf conditions, the media-driven pre-conditioning of wants and expectations, and various playing companions have on our experience with any golf course and any round of golf -- humbly proposes for your half-hearted consideration a list of 10 golf courses that after some (indeterminate) amount of post-round reflection he has come to believe he enjoyed the most on the days he played them, and making no claims whatsoever, in whole or in part, implicitly or explicitly, regarding the validity and objective (or even instructional) value of this list for any other golfer currently (or soon to be) alive, on this planet Earth or on any other planet in the known universe."

Agreed? Is that what such ratings/rankings are for you? Great. I will then await the next such list with bated breath....          

[Ulrich Mayring]

The category method gives an explanation of the number arrived at. Without categories you are just pulling a number out of a hat and there is no way to qualitatively judge the result.

The problem with tallying up a lot of subjective numbers to arrive at a seemingly objective result is worse without fixed categories. Because you are tallying up numbers, which use all kinds of (unspoken) categories. If you want to tally up, say, the 100 numbers from 100 players, then they are way more comparable when they're based on 4 categories rather than 200, of which not a single one is used by all 100 players.

The truth is that everyone is using categories, but not everyone is putting them out in the open, thereby making himself verifiable.

Ulrich

[Sean_A]

Ulrich


I think the real truth is that people have ideas of what they think are the best courses and then use categories to justify those ideas.  Be honest, do you ever change your ranking when you realize any given course comes out higher or lower than where you believe it should have before you went through the process of ranking?  Do you look at your ranking and think, wtf, that ain't right...I better right the ship...then go back and alter category scores? I bet you have....why...because a numerical system may be te ebst you can come up with, but it is far from perfect and often leaves something to be desired when we do an eye check. 

My categories for quality are twofold; course and beauty.  I don't break it down any further than that.  Sure, I try to determine what sort of course it is and what was the intention of the design and judge it based more on that idea, but it still comes down course and and beauty.

Pietro
 
Ranking courses based on anyone's idea of quality is inherently unsatisfactory.  I have said it a million times...I am far more interested in the favourites of folks because I think I have a better chance of receiving an undiluted opinion rather than one filtered through the lenses of ranking. 

Ciao

[Jud_T]

Ulrich,

So you're just pulling numbers out of a hat for each category and adding them up.  There's no such thing as an impartial judge in this unless you're talking about length, scoring average etc.  There's no sense in comparing courses of varying types against one another.  Trying to jam The Sacred Nine and Carnoustie into the same list and implying that one is truly better is a fool's errand.  If you want to rank courses by type (pretty, challenging, fun) that would make a lot more sense and be much more useful to players. (Thanks Mike Nuzzo).

[Thomas Dai]

[size=78%]  [/size]
My categories for quality are twofold; course and beauty.

Interesting. Say for example, Troon vrs Turnberry. Course vrs course, less beauty vrs lots of beauty?


Atb

[Jim Hoak]

Sean--Why beauty?
I think more harm is done in golf course architecture by focusing on pure beauty than any other thing. Certainly as beauty affects play, a fun experience, etc. it needs to be considered.  But by itself trying to achieve high beauty in a course detracts from its success as a course.  Beauty might mean more trees, flowers, waterfalls, fancy bridges, etc., but does that enhance the worth of a course?
I'm sure you meant something other than what I'm thinking, and I defer to your experience.  Please amplify on your thinking on beauty as a core determinant of a course's value.

[Jim Hoak]

I had the same thoughts in my mind as Thomas--Troon would be low on the beauty scale; Loch Lomond would be high.  But I'll take the "ugly" one.

[Sean_A]

Troon wouldn't be low on my beauty "scale".  Its quite an attractive property which is far more beautiful than 95% of the courses on the planet.  In any case, beauty is a minor "category" compared to course.  That said, sometimes beauty can work against a course if showcasing the views is an over-riding factor in design decisions.


Ciao   

[Ulrich Mayring]

So you're just pulling numbers out of a hat for each category and adding them up.

Jud,

True, but I am using the same four hats every time. And because I do that, I can compare. I agree it is quite difficult to say whether Royal Worlington is better or worse than Carnoustie. But it is much easier to say which course has more quirk or the better scenery or better flow. Not publishing your categories is a bit of a cop-out and if Tom Doak hadn't made it politically correct, we might not even discuss it

Sean,

Of course I make mistakes. And I correct them. Sometimes I change my ranking two years after playing a course, because it lingers in the back of my mind and I find my initial evaluation to be lacking.

That being said, I never change categories in order to enhance or diminish the total number. That would defeat the entire purpose of using categories - I am using them precisely because the total number becomes more sensible and verifiable that way.

I have nothing against using just one number, but I believe you should make the effort to explain what this number means. Otherwise, if you think your criterion is not explainable or changes all the time, you should stick to writing textual reviews and not give a number at all. I am doing both, because I believe both approaches have merit and can be done respectably.

Ulrich

[Mark Pavy]

If you could design the Top 100 Courses rankings criteria for Golf Digest, what would your categories be?

1. Is the Golf Digest certificate displayed in the foyer. 10%
2. Is there a copy of Golf Digest in the foyer. 10%
3. Is there a copy of Golf Digest in the Pro-Shop. 10%
4. Is there a copy of Golf Digest in the restrooms. 10%
5. Is there a copy of Golf Digest in the bar area. 10%
6. Is there a copy of Golf Digest in the machinery shed. 15%
7. Does the Club/Course use the Golf Digest ranking in marketing material. 30%
8. Is there a golf course. 5%