New ranking criteria

[jeffwarne]

many have been critical of GD's ranking methodology and raters.
Mark says it's simply a reflection of one man's taste
Some very good comments, and no doubt many well intentioned raters.

What would be the ideal criteria?
Note-I have no set criteria and am very biased to subjectivity, but that's certainly no way to run rankings

My stab.
25% 1.Routing---suggestions-  intimacy, isolation, flow and rhythm, variety of areas of the property visited, wind , walkability. beer loops
20% 2. variety--suggestions--terrain encountered, hole length, uniqueness of landforms both created and natural, memorability
15% 3.setting--suggestions-scenery, interaction with structures/town/clubhouse, natural views both on and off course, charm
25% 4.Challenge,strategy, imagination,---no suggestions but conditioning would affect all 4
15% 5. playability and fun-suggestions wide corridors, minimal ball hunting



Suggestions/corrections/additions/omissions welcome

[Jason Thurman]

I rate your criteria a 3.64.

It's incredibly sad to me to see guys on this site yearning so deeply to have a published list of subjectively evaluated courses that validates their own subjective tastes. Why isn't it enough to just decide what courses we like all on our own?

There is no objective truth lurking out there. I can understand how that's a depressing thought to a guy who spent years of his life and thousands of dollars traveling around and playing all the courses that the magazines told him to play, and why he would be so angry about it now that he's realized how stupid his goal was. But for the rest of us, what do we really want from these things?

[jeffwarne]

I rate your criteria a 3.64.

It's incredibly sad to me to see guys on this site yearning so deeply to have a published list of subjectively evaluated courses that validates their own subjective tastes. Why isn't it enough to just decide what courses we like all on our own?

There is no objective truth lurking out there. I can understand how that's a depressing thought to a guy who spent years of his life and thousands of dollars traveling around and playing all the courses that the magazines told him to play, and why he would be so angry about it now that he's realized how stupid his goal was. But for the rest of us, what do we really want from these things?

Damn Russian judges
I'm going to do like GD and throw that 3.64 out

Jason,
Your favorite course is? and why
It's an exercise on an architectural discussion board, not a dogma

and mine IS very subjective, but no doubt my subjectivity has been influenced byobjectively  reading, visiting, playing, traveling,speaking with architects, and working on golf courses

[Mark Bourgeois]

Jeff,

And maybe this is how you did it, but what if you thought about this in terms of "cohort groups"? Think about the, say, three attributes your considered best courses all possess. They may not possess in equal degrees, so don't bother weighting the attributes. The attributes could be at odds with each other, too, like "fun" vs "resistance to scoring."

List those here then ask: which courses fit these attributes the best?

[BCowan]

25% 1.Routing---suggestions-  intimacy, isolation, flow and rhythm, variety of areas of the property visited, wind , walkability. beer loops
20% 2. variety--suggestions--terrain encountered, hole length, uniqueness of landforms both created and natural, memorability
15% 3.setting--suggestions-scenery, interaction with structures/town/clubhouse, natural views both on and off course, charm
25% 4.Challenge,strategy, imagination,---no suggestions but conditioning would affect all 4
15% 5. playability and fun-suggestions wide corridors, minimal ball hunting

15% 3.setting--suggestions-scenery, interaction with structures/town/clubhouse, natural views both on and off course, charm

I'd go with charm and nuances and bump it to 20%

Another great thread

[BHoover]

I rate your criteria a 3.64.

It's incredibly sad to me to see guys on this site yearning so deeply to have a published list of subjectively evaluated courses that validates their own subjective tastes. Why isn't it enough to just decide what courses we like all on our own?

There is no objective truth lurking out there. I can understand how that's a depressing thought to a guy who spent years of his life and thousands of dollars traveling around and playing all the courses that the magazines told him to play, and why he would be so angry about it now that he's realized how stupid his goal was. But for the rest of us, what do we really want from these things?

I agree completely, Jason. I have my own list of courses that I love and those that I don't. My preferences are not based in any way of published rankings. Ranking is inherently subjective, period.

[jeffwarne]

25% 1.Routing---suggestions-  intimacy, isolation, flow and rhythm, variety of areas of the property visited, wind , walkability. beer loops
20% 2. variety--suggestions--terrain encountered, hole length, uniqueness of landforms both created and natural, memorability
15% 3.setting--suggestions-scenery, interaction with structures/town/clubhouse, natural views both on and off course, charm
25% 4.Challenge,strategy, imagination,---no suggestions but conditioning would affect all 4
15% 5. playability and fun-suggestions wide corridors, minimal ball hunting

15% 3.setting--suggestions-scenery, interaction with structures/town/clubhouse, natural views both on and off course, charm

I'd go with charm and nuances and bump it to 20%

Another great thread

Thanks Ben

They are merely marked "suggestions" following the 5 criteria, allowing ample room for subjectivity in each "objective" category.
each "rater" could interpret each category however they wanted

Brian,
no one's asking you to base your preferences on published rankings.
I'm simply asking people that object to GD's criteria (and there are many evidently) for THEIR criteria, and what they would do differently than GD..

Surely you have a reason why a course/courses is your favorite.
Try it, you'll like it

You clearly have objective criteria based on your statement that you would "steer clear of Goat Hill" because you found the pictures "bland  and boring" and the condition a turnoff.
the pictures in that thread CLEARLY illustrate heaving terrain, extreme in places, multiple blind shots, and steeply tilted/sloped greens, often away form the line of play, and the illustrated and commented on conditions were ultra fast and firm.

So I'd say you enjoy
1.flat
2.visible
3.greens without slope and tilt
4.green and soft conditions
5. "bunkers and fountains to aim at"

not defending the Goat, just saying you do have objective criteria that you used to say the Goat was lacking.
of course I extrapolated abit

[BHoover]

Jeff, I understand your question. I wasn't trying to squash your premise. I should have been clearer--I'm not sure why I love the courses I love and hate those that I hate. I guess it's just one of those "you know it when you see it" types of things. I'm sure I could get more philosophical or technical, but I'm not sure that's productive. But I certainly did not mean to suggest that your question is without merit. If that was the implication, I'm sorry.

Giving your criteria a second look, I think the criteria have great merit...items 1 and 4 are probably what I look for more than anything else. The courses I tend to like probably share those characteristics. Actually I prefer variety too!

You have proved the flaw in my logic...I concede!  

[jeffwarne]

No apology needed
see my above edited post.
as one who always say they are subjective (me) I'm just saying it's interesting to quantify where our "subjectivity" came from as I did with your (initial) reaction to pictures of the Goat (which is indeed often polarizing)

Chances are still pretty good you wouldn't like the Goat (for different reasons )

[Michael George]

Jeff:

I applaud your efforts.  This exercise shows how difficult it is to rank courses.  And as I have said many times before, I don't want to tell someone how they "have" to view golf courses, even though I may have my views.

So what are my views:

Here are some criteria that I use to determine my likes/dislikes of golf courses - many of which are similar to Jeff's criteria:

(1)  Mental Challenge/Pleasurable Excitement - how much did the golf course make me think about each shot.  Golf is best played between your ears.  If choices are made for you by the architect, the course lacks.  Further, how many shots provided real excitement to the player.  Hitting the shot on #16 at Cypress has to count for something......and not just the challenge of the shot.

(2)  Challenge versus Playability - how much does a course challenge the best players, yet remain playable for the lesser players.  A course needs to provide both.  It is not a decision between one and the other so I differ from Jeff in separating the 2 criteria.  

(3)  Environment - call it ambience or memorability or charm or whatever.  A rater should consider the geographic setting of the golf course and the beauty of the golf course. However, it is not just geography and beauty for this category.  How does the clubhouse setting compare to the course?  What is the vibe of the course/club?  For instance, a review of Sand Hills may provides points for the beautiful sandhills setting and the wonderful unpretentious nature of the club, while also possibly detracting for the detachment between clubhouse and course.  A review of Winged Foot Golf Club may not have the geography and views of other courses, but the vibe of one of the oldest and storied clubs is incredibly high.  Playing the 18th at WFW and knowing the history on that hole is special and to not account for this is a mistake.  

(4)  Conditioning/"Ground Architecture" - I think Golf Digest gets conditioning right currently, even though some raters may not apply it correctly.  How firm and fast is the golf course?  However, I would also add to this criteria, how much does the ground impact play?  Golf is best played both in the air and on the ground.  If the ground does not impact a shot, then it is less interesting.  Chipping areas and false fronts and other features provide wonderful fun shots for players.  It does not need to be fescue and links golf for the ground to impact play.  See Pinehurst #2 for an example of the ground really impacting a parkland course.  

(5)  Variety - I think Jeff properly covered this criteria.  A great golf course remains enjoyable to play after repeated rounds.  Monotony bores the golfing soul.

(6)  Routing - again, I think Jeff properly covered this criteria.  If a rater prefers walking, this is the category to hammer the cartball courses.

Just some of my thoughts.

[Garland Bayley]

I shamelessly steal from the good Dr.

I had the pleasure of playing Bude and North Cornwall during my visit to England for Buda Cup this year. I played it the day after I played Saunton, and I found it to be much more pleasurable than playing Saunton East in particular, but also than playing Saunton West.

In evaluating a course, I have found that Alister MacKenzie's preferences from The Spirit of St. Andrews match my preferences to a high degree, some more than others. I have made a synopsis of his preferences and assigned a rough relative weight to them below.

1 Two loops of nine holes
3 Large proportion of good two shot holes and 1 at least four one-shot holes
1 Short green to tee walks
3 Greens and fairways should be sufficiently undulating.
3 Every hole different in character.
1 Minimum of blindness in approach shots.
2 Artificial features should be indistinguishable from natural features.
2 Sufficient number of heroic carries from the tee, but always with alternative route for the high handicapper.
3 Variety of strokes to play various holes, i.e., every club.
3 Complete absence of the need to search for balls.
2 So interesting the best will be motivated to develop shots not previously capable of.
2 Should allow wide detours for high handicappers to avoid hazards.
2 Consistent presentation throughout year with approaches having the same consistency as the greens.

As viewed from Google Earth, one can see there will be a problem with some green to tee walks. What cannot be seen from such a view is the positive brought by the undulations present in the course, and the negative from some blind approaches.

[Lester George]

Wow, Jason!  Nicely said.

Lester

[jeffwarne]

Wow, Jason!  Nicely said.

Lester

I don't understand, and while you may agree with his disdain of objective analysis, it was hardly nice. .
The point was for those who had issues with GD's methodology, and was merely an exercise for me to attempt to quantify my own typically subjective views-and invite others to add, subtract or edit.
After all what did Mackenzie know anyway......
but then as I stated, maybe I just don't understand-or simply missed the  ---

[Chris Shaida]

Wow, Jason!  Nicely said.

Lester

I don't understand, and while you may agree with his lack of objective analysis, it was hardly nice.
but then as I stated, maybe I just don't understand

It's quite possible that Lester was being ironic since pretty clearly Jason's comments, while certainly interesting and quite possibly cogent, were not 'nice' in either the old sense of the word of 'finely wrought, subtle' nor the current defacto meaning of 'blandly pleasant'!

So Jason, taking your point about the bad consequences of ranking (people don't think for themselves, etc.) don't you think that once you decide you DO like something the reasons that YOU have for liking that are worth exploring?  That is, leaving comparative ranking aside, if you were talking to someone and said that you really liked course X and they asked why would't you then use something like the categories that Jeff suggested to explain what you did (and didn't) like about that course?

[jeffwarne]

Why don't the magazines just require the raters to maintain a spreadsheet of all eligible courses they've ever played in order from top to bottom and let computer algorithms like Sagarin or whatever determine which course beats how many other courses on those lists and where the " common opponents" are because there's bound to be plenty with the whorish tendencies of raters, and do the top 100 that way?  Let the damn computers figure it out based on raw comparative data instead of artificial numerical data?  

Yes, some guys have played a better list of courses and some worse, but that goes to "strength of schedule" for each Rater so one rater's 12th best would not be weighted the same as another's.  So if I have Pasa as #63 on my list, it has W's for however many eligible courses I say it's better than and 62 L's to the 62 courses on my list that I say it's worse than.  Add up all the Ws and L's from all the raters and adjust for strength of schedule based on common opponents and opponents of common opponents as many times as you need to get it weighted properly, and you've got yourself a list.  Did any of the geniuses at the mags ever think of that?  

This whole exercise is silly.  

but not so silly that you couldn't resist coming up with 2  paragraphs of objective criteria of your own.

[Garland Bayley]

...Did any of the geniuses at the mags ever think of that?  

This whole exercise is silly.  

Isn't that similar to the path Tom Doak started Golf Magazine down?

[Paul Gray]

100% sustained fun, meaning you can enjoy it as much, if not more, on your hundredth play as on your first. And that is all there is, since golf is supposed to be fun. 

[Sean_A]

Jeff

I will bite.  I don't really know percentages for this kind of stuff and I am not sure they are or would be static.  Sometimes I come across stuff which really elevates a course even though there is some mundane stuff to wade through.  So....



1. Lets call it routing for lack of a better term: the quality of the walk, the views, green sites and use of natural features.  Cart courses are starting in a big hole so far as I am concerned...its not necessarily irrevocable, but something special has to come down the pipe. 

2. The site: which really boils down to terrain and quality of grasses/soil. 

3. The greens: when I say greens it is mostly about firm, true greens.  I am not overly bothered about speedy greens or terribly bold contours.  Sure, I want greens with interest; some contour, some tilt with the odd crazy surface, but for the most part, variety, firmness and well draining are the elements I prize most.

4. Man-made Features: this would include bunkering, bumps, hollows, water etc.  The main aspects I look for are placement, economy in design and balance. 

Ciao

[Dave Doxey]

I've always thought that ranking using numerical scoring is not the best approach.

A statistical method of head-to-head ranking of courses played by each reviewer (Pairwise Comparisons?) would be a better approach, given that all raters do not play all courses.

Use a ranked list, of only those courses played, from each reviewer.  The lists then can be used, with statistical methods, to produce an overall ranked list.

That said, having a set of guidelines for the raters to consider when doing their rankings is a good thing.

[Joe Hancock]

100% sustained fun, meaning you can enjoy it as much, if not more, on your hundredth play as on your first. And that is all there is, since golf is supposed to be fun. 

Is recreational golf the version that's supposed to be fun, or all golf, including tournament golf?

Joe

[Jason Thurman]

and mine IS very subjective, but no doubt my subjectivity has been influenced byobjectively  reading, visiting, playing, traveling,speaking with architects, and working on golf courses

You clearly have objective criteria based on your statement that you would "steer clear of Goat Hill" because you found the pictures "bland  and boring" and the condition a turnoff.

I don't understand, and while you may agree with his disdain of objective analysis, it was hardly nice. .

but not so silly that you couldn't resist coming up with 2  paragraphs of objective criteria of your own.

[Jason Thurman]

So Jason, taking your point about the bad consequences of ranking (people don't think for themselves, etc.) don't you think that once you decide you DO like something the reasons that YOU have for liking that are worth exploring?  That is, leaving comparative ranking aside, if you were talking to someone and said that you really liked course X and they asked why would't you then use something like the categories that Jeff suggested to explain what you did (and didn't) like about that course?

Chris, I absolutely think it's worth discussing the reasons why one likes or dislikes a given course. I just think it's silly to try to reduce one's impression of any and all courses down to a series of subjectively assigned numerical values across a series of subjectively selected categories.

To use some examples from other arts, some songs are great because they have great melodies while others are great because they have great harmonies, beats, lyrics, instrumentation, and/or any combination of those factors and other factors. Movies can be great because they're funny, because they're beautifully filmed, because they shed light on truths of humanity, because they reveal new perspectives on events or issues, because they have great dialogue, and/or any combination of those factors and other factors. It would be silly and reductionistic to evaluate songs as 20% melody, 30% instrumentation, 15% lyrics, etc., and it would be equally silly to evaluate cinema in a similar manner, just as it's silly to try to reduce the numerous ways that a golf course can achieve something special down to a mathematical formula of X% routing, Y% variety, etc.

If you're putting together a list for Golf Digest or Buzzfeed or just want something to talk about, rating criteria can be  a fun topic of debate. But to pretend any formula that any of us will come up with will approach anything resembling an objective truth is, again, silly. Courses can be great for all kinds of reasons, and it's far better to evaluate them on their own merits rather than evaluating them on their adherence to a set of arbitrary criteria.

[BCowan]

To use some examples from other arts, some songs are great because they have great melodies while others are great because they have great harmonies, beats, lyrics, instrumentation, and/or any combination of those factors and other factors. Movies can be great because they're funny, because they're beautifully filmed, because they shed light on truths of humanity, because they reveal new perspectives on events or issues, because they have great dialogue, and/or any combination of those factors and other factors. It would be silly and reductionistic to evaluate songs as 20% melody, 30% instrumentation, 15% lyrics, etc.,You just broke a movie down, just as Jeff did.  How is that any different?  Just because he put percentages down, that is the only difference.  Do you know how movie ratings are done?  Maybe if they were done to a system like Jeff's, Caddyshack wouldn't of gotten 2 stars when it came out .  How would you compare songs in a head to head matchup? personally, i think lyrics are little low in your system .  For a guy who loves shotlink and other stats, you now are changing ur tune   to Art form.  

 and it would be equally silly to evaluate cinema in a similar manner, just as it's silly to try to reduce the numerous ways that a golf course can achieve something special down to a mathematical formula of X% routing, Y% variety, etc.Says the guy who loves shotlink and stats.

If you're putting together a list for Golf Digest or Buzzfeed or just want something to talk about, rating criteria can be  a fun topic of debate. But to pretend any formula that any of us will come up with will approach anything resembling an objective truth is, again, silly. Courses can be great for all kinds of reasons, and it's far better to evaluate them on their own merits rather than evaluating them on their adherence to a set of arbitrary criteria.merits and criteria same thing?

[Josh Stevens]

What is it with the intimacy and isolation requirement?  That surely is more of a subjective aesthetic criteria rather than being related to architectural merit.  Few properties have that space. TOC, N Berwick and Royal Melbourne would fail that criteria as everyone on course can see you make a fool of yourself

[BCowan]

Josh,

   Great point, I missed that when I first read it.  ''flow and rhythm, variety of areas of the property visited, wind , walkability. beer loops''

Wind, walk ability, and beer loops caught my eye first