Why do we rank? Why do we CARE to rank?

[Ronald Montesano]

...and a third question: is the thread title composed of two distinct questions or is it simple semantics?

I compose Buffalo-Niagara rankings of public-access and private courses for my regional website, to foment discussion among viewers/readers. Personally, lists were a help to me as I navigated the early waters of GCA, but being given a number on some architect's list or a space in some magazine's rankings didn't affect me as much as what was said about the golf course. I like the eclectic and the unique, and find this comes not from a number but from the description and the narrative.

What purpose(s) does ranking courses specifically by number (at times, to the 3rd or 4th decimal point) serve?

[Brent Hutto]

Under advice of counsel, I have no comment.

[Jason Topp]

I generally enjoy golf courses.  But saying that I enjoyed a golf course doesn't lead to a very interesting discussion.  The primary value of ranking from my perspective is that it forces you to answer the question - which is better?  When one is forced to answer that question, it raises the question of why and hopefully helps to draw insight from the process.

The main reason one sees rankings, I presume, is that it sells magazines, newspapers and, more likely, advertising.

[Thomas Dai]

Comparisons and rankings may have their faults but they are generally indicative of standards, whether that standard be excellent, okay, poor  or whatever. Without comparisons and rankings the golfing taste buds of many folk, me included, would not have been awakened to the same extent and thus we may not have travelled to many interesting and enjoyable courses outwith the immediate area. Whilst not trying to tick off course lists, there are many courses that I've played that I would never have travelled to without mention in a comparison or a ranking and thus many thoroughly enjoyable and fondly recalled trips/holidays would not have taken place and many regions would have been unexplored.
Comparisons and rankings also allow for debate and discussion which is frequently very enjoyable, especially in relation to planning future trips
ATB

[Jud_T]

I would give this thread a Doak 4.136.  A bit repetitive; let's face it, we've all seen these threads before, but what the hell else are you going to talk about these days that doesn't involve Merion or isn't completely OT?  A thread worth perusing in a pinch. but not worth going out of one's way to see.

[Doug Ralston]

Hey, mine is bigger than yours! Who cares? I don't, I just wanted YOU to admit it! 

Doug

[RJ_Daley]

I would give this thread a Doak 4.136.  A bit repetitive; let's face it, we've all seen these threads before, but what the hell else are you going to talk about these days that doesn't involve Merion or isn't completely OT?  A thread worth perusing in a pinch. but not worth going out of one's way to see.

I would give this thread a Doak 4.135 because it doesn't mention the need for people's validation of their own tastes relative to others, and need to be thought of as relevant.  

Actually, in my view, Jud and Ron hit upon the idea of the futility of a ranking that places one course in some form of ranking consensus, a fraction of a point above another.  That is truly ridiculous.  To say a very fine course in one region is .01 better than a course in another region that is only defined as "best modern" or "best classic" or "best world" is futile without narrative.  

Thus, in my mind, I think the general categories are good for placing courses "in class"; like best modern, classic, world, etc.   But, then I think there should merely be another category, like "among the very or elite best", "among the best". "among the regionally good", "sub-standard", and "not very good".  

Then, when one compares a Pine Valley to a Cypress Pt, or Royal Melbourn in the top category of say, "Elite World", one can go to narratives to see how competent golf architecture and player experts "describe them in specific terms".  There wouldn't be so many raters that way.  It would be down to people that can write a narrative and demonstrate their competence in areas of evaluation and architecture-design standards.  A narrative exposes the evaluation experts competence and depth of knowledge far more than a numerical 4.133. IMHO.

[Tom_Doak]

The only reason I came up with the Doak scale rankings for The Confidential Guide, was as a foil for what I'd written in the reviews.

I didn't want to have to be "fair and balanced" in every review, because that wastes a lot of space.  So, I wanted to be able to criticize great courses in the review, but still post a number that indicated I didn't think the course was crap ... a number that you could quickly compare to my numbers for the other courses.

When the book was only available to 40 friends, not many of them made a lot of comments about those ratings.  But when the book went public, the numbers became the focus of discussion.  I think that says a lot about human nature, but it also says a lot about how magazines and other writers in particular seek to stir up controversy.

[Jud_T]

RJ,

I think it sells magazines and is merely the result of mathematically tabulating a large pool of raters to a given set of criteria.  A better example is a trusted individual, like the Confidential Guide.  Based upon the juxtaposition of a known entity's predilections and one's own, it's fairly straightforward to go into the detailed text and, for example, determine that one may prefer to play Westward Ho, a 6, than Shadow Creek at a 9.

[Tom_Doak]

Actually, in my view, Jud and Ron hit upon the idea of the futility of a ranking that places one course in some form of ranking consensus, a fraction of a point above another.  That is truly ridiculous.  To say a very fine course in one region is .01 better than a course in another region that is only defined as "best modern" or "best classic" or "best world" is futile without narrative.  

Thus, in my mind, I think the general categories are good for placing courses "in class"; like best modern, classic, world, etc.   But, then I think there should merely be another category, like "among the very or elite best", "among the best". "among the regionally good", "sub-standard", and "not very good".  

Then, when one compares a Pine Valley to a Cypress Pt, or Royal Melbourn in the top category of say, "Elite World", one can go to narratives to see how competent golf architecture and player experts "describe them in specific terms".  There wouldn't be so many raters that way.  It would be down to people that can write a narrative and demonstrate their competence in areas of evaluation and architecture-design standards.  A narrative exposes the evaluation experts competence and depth of knowledge far more than a numerical 4.133. IMHO.

RJ:

As the guy who introduced numbers to the ranking system, I feel compelled to respond to that.

Before GOLF Magazine started ranking the courses in order, the GOLF DIGEST way was to group courses as "First Ten" or "Second Ten" or whatever.  Then they would sit around the smoke-filled room every couple of years and decide to move a course down a notch and another up a notch, based on their own recent impressions, to give people something to discuss.  There was no method behind it at all.

Getting the panelists to evaluate the courses and actually using those evaluations only started in the 1980's.  Once you had that data, it was easy to rank the courses, and silly to obscure the data in broad categories like you are advocating, above.

The problem is NOT the data, it's what people take away from it.  You are dead right that the difference between 20th place and 21st place is ridiculously small, and really meaningless.  As far as I know, no ranking has ever disputed that point.  But let's extend the argument -- is the 0.12 difference between 20th place and 47th place meaningful?  If it's not, then the entire rankings are a meaningless exercise [which is certainly one possible conclusion].  But if at some point, people's evaluations become meaningful, then how do you draw the cut-off lines, when the difference between one course and the next is always .01 or .003 or something very small?  Note that the biggest distinction the lists make are between 100th place and 101st (not listed), and the difference there is smaller than the gaps between courses at the top of the chart.

In the end, the small differences between the courses are pretty meaningless.  If everyone just accepted that, then listing them in order would be seen as the only fair thing to do.

[Greg Tallman]

While 1/100th of a "point" may be meaningless it can be the difference between being "Top 100" and jettisoned into the abyss. Two years ago I found that 1/100th of a point rather important. 

[john_stiles]

I would rank this site as #2

[john_stiles]

But, maybe like Jud above, I enjoy reading about courses a little more than seeing a number.

The only number that shocked me was the zero Tom Doak handed a course in Georgia.  I just put that course as one I would play if friends invited me, or maybe a GCA outing.

So in that sense, I would prefer the older, grouping in alphabetically order.

My leaning is towards a Ran, GCA w/photos, Max, or Doak review.

Oh yeah,  another shocking number was a southern course that got a 10 and a phone call.

[Jim Nugent]

To say a very fine course in one region is .01 better than a course in another region that is only defined as "best modern" or "best classic" or "best world" is futile without narrative.  

Thing is, that's not what those rankings say.  As I understand it, they simply RANK the courses.  They don't say how much better one course is than another.  There could be a huge difference between the courses ranked 9 and 10 in the raters' minds.  Or there could be next to nothing.  

One example of this is the Mohs hardness scale for minerals.  Diamonds are ranked 10, the hardest mineral known.  Rubies are 9, and talc is 1.  But diamonds are not 11% or so harder than rubies.  They are four times harder.  They are 1600 times harder than talc.  The Mohs scale completely obscures that, though.  Just like saying Tiger was the #1 golfer in year 2000 obscures how much better he was than #2 and everyone else.  

Golf Magazine and Golf Week, I believe, make it even harder to draw absolute conclusions because they rank the courses in groups.  To get a score of 10, a course must be among the top 3 (or 5 or whatever) in the world.  That doesn't even tell you how the courses rank within the category.  An example would be if the Mohs scale gave a score of ten to the two hardest minerals.  That would rank diamonds and rubies the same, even though diamonds are in fact four times harder than rubies.  

The Doak scale does the same thing.  Maybe I'm wrong, but I doubt Tom personally considers all the courses he ranked 10 to be as good as each other.  He probably considers some better than others.  Same with the 9s, 8s and other scores in his guide.  But there's no way to know.  

How much better does Tom consider Pine Valley than Shadow Creek?  About 11%?  I don't believe there's anyway to know from the CG.  Same is true of the rankings in GM and GW.  

As for Ronald's question: people rank almost everything.  Restaurants.  Hotels.  Cars.  Kitchen appliances.  Computers.  TVs.  Singers.  Songs.  Guitarists.  Guitar riffs.  Actors (academy awards).  Football teams.  Golfers (Tiger, Jack or Bobby?)  Golf course architects (Ian Andrews' interesting list and articles are one example).  The list goes on and on.  

Who is the best?  Ranking tries to establish that.  When there's an objective measure -- like running 100 meters -- it's not so hard.  When things are subjective -- like art, music or golf course architecture -- we can have lots to talk about.  

[Mark Chaplin]

Jim you but the nail on the head, we rank everything. Best meal I've had for a while, the 2000 Bordeaux wasn't as good as the 1990. It goes on and on, numbers just help to add order to it.

[A.G._Crockett]

We rank courses for the same reason we rank teams and keep score in games.  Human beings are for the most part competitive and winning and losing.  Lombardi said something to the effect that the greatest human emotion is winning and the second greatest is losing.  Those drives lead us to rank restaurants, movies, wine, and golf courses.  It's in our DNA.

[Jason Thurman]

Thus, in my mind, I think the general categories are good for placing courses "in class"; like best modern, classic, world, etc.   But, then I think there should merely be another category, like "among the very or elite best", "among the best". "among the regionally good", "sub-standard", and "not very good".  

RJ, while I think such a system would probably be more accurate and more accommodating of the true subjectivity of ranking, I also think it would ultimately suck. The best thing about rankings is that they give us plenty of fodder to argue about. Acknowledging in the rankings themselves that course preferences can't be distilled to a science also makes them less interesting and fun to debate.

Then, when one compares a Pine Valley to a Cypress Pt, or Royal Melbourn in the top category of say, "Elite World", one can go to narratives to see how competent golf architecture and player experts "describe them in specific terms".  There wouldn't be so many raters that way.  It would be down to people that can write a narrative and demonstrate their competence in areas of evaluation and architecture-design standards.  A narrative exposes the evaluation experts competence and depth of knowledge far more than a numerical 4.133. IMHO.

I agree about a narrative component. I played Pinehurst #2 with my mother last year, and we got paired with a Gold Digest rater. We really enjoyed him, and she mentioned after the round that she wanted to read some of his ratings. When I explained to her that he really just uses a number system and doesn't write any sort of narrative for the magazine, she thought that sounded pretty stupid. I agree, especially in 2014. You can get lots of clicks for written reviews these days. People love reading them. I predict the magazine that starts publishing short narratives from ALL raters explaining why they give one score or another will get a lot of page views. I'd read all day.

[Steve Lapper]

Thus, in my mind, I think the general categories are good for placing courses "in class"; like best modern, classic, world, etc.   But, then I think there should merely be another category, like "among the very or elite best", "among the best". "among the regionally good", "sub-standard", and "not very good".  

RJ, while I think such a system would probably be more accurate and more accommodating of the true subjectivity of ranking, I also think it would ultimately suck. The best thing about rankings is that they give us plenty of fodder to argue about. Acknowledging in the rankings themselves that course preferences can't be distilled to a science also makes them less interesting and fun to debate.

Then, when one compares a Pine Valley to a Cypress Pt, or Royal Melbourn in the top category of say, "Elite World", one can go to narratives to see how competent golf architecture and player experts "describe them in specific terms".  There wouldn't be so many raters that way.  It would be down to people that can write a narrative and demonstrate their competence in areas of evaluation and architecture-design standards.  A narrative exposes the evaluation experts competence and depth of knowledge far more than a numerical 4.133. IMHO.

I agree about a narrative component. I played Pinehurst #2 with my mother last year, and we got paired with a Gold Digest rater. We really enjoyed him, and she mentioned after the round that she wanted to read some of his ratings. When I explained to her that he really just uses a number system and doesn't write any sort of narrative for the magazine, she thought that sounded pretty stupid. I agree, especially in 2014. You can get lots of clicks for written reviews these days. People love reading them. I predict the magazine that starts publishing short narratives from ALL raters explaining why they give one score or another will get a lot of page views. I'd read all day.

Rumor has it that Golf Magazine Panelist may soon do just that.

[Tom_Doak]

I played Pinehurst #2 with my mother last year, and we got paired with a Gold Digest rater. We really enjoyed him, and she mentioned after the round that she wanted to read some of his ratings. When I explained to her that he really just uses a number system and doesn't write any sort of narrative for the magazine, she thought that sounded pretty stupid. I agree, especially in 2014. You can get lots of clicks for written reviews these days. People love reading them. I predict the magazine that starts publishing short narratives from ALL raters explaining why they give one score or another will get a lot of page views. I'd read all day.

God bless you.  God help you.  I have read short narratives from many panelists about my own work and about other famous courses, and the more I've read, the more I wonder how many of these people are qualified to rate courses.  Some of the short comments are just frighteningly inaccurate ... not just opinion but on the very facts of the courses.

[Brent Hutto]

I work with numbers for a living and it is truly amazing how much credibility a piece of rank bullshit achieves by simply being expressed as a number, preferably a number with a couple of digits past the decimal point.

There's an old saying in my line of work, "Everyone believes the numbers except the guy who collected them".

[Tom_Doak]

I work with numbers for a living and it is truly amazing how much credibility a piece of rank bullshit achieves by simply being expressed as a number, preferably a number with a couple of digits past the decimal point.

There's an old saying in my line of work, "Everyone believes the numbers except the guy who collected them".

  I haven't heard that one before, but I can vouch for it in this field!

[BCrosby]

TD says:

"I find golf holes on maps and visualize how they will work, not what they will look like."

Bravo. Tom captures (perhaps inadvertently) one of the shortcomings of discussions here and elsewhere about golf architecture. The look of bunkers, greens and tees can suck the air out the room when people talk about golf courses.

It was not always that way. Digging through old magazines in the 1920's and earlier, how people talked about the quality of a hole was most often demonstrated by simple illustrations showing the location of its architectural features. How "the hole worked" was what people were trying to articulate through these drawings. The beauty (or not) of the hole was rarely a main consideration.

The weight given to those two ways of talking about a hole has flipped in recent years. The aesthetics of a hole seems to be the first topic discussed and only sometimes does the conversation move on to how the hole plays. Which is what it means to hollow out conversations about the hole's architecture. I think that course ratings in mass circulation golf mags are (and have been) a big part of that problem.

Bob  

 

[C. Squier]

Hey math majors, the more people you have voting, the deeper the decimal points will become. It has nothing to do with singular panelists, but a law of large numbers.

[Brent Hutto]

Averaging two tons of nonsense together does not give you more meaningful numbers than you'd have for two pounds of nonsense.

If individual panelists are by and large incorrectly perceiving, recalling or judging the on-the-ground facts about a golf course then averaging a large number of their ratings gives you a very precise number that's no more accurate a reflection of reality than the individual numbers represented.

Yet it's an easily observed fact of human nature that when presented with a number like "3.224 points" and claiming it is an average from "216 raters" will be treated as Gods Own Truth Down From The Mountain even by people who would dismiss "3 points" assigned by a single rater as meaningless.

First show me one, individual rater who can be proven to assign correct numbers reflecting the actual reality of a golf course. That's a tall order. Then I say OK, I don't want one rater who I can trust to be unbiased, insightful and...you know...CORRECT about the course. I want one hundred such reliable, known to be correct raters so I can average them together. That's a much taller order indeed.

[Jud_T]

The problem is that even if you had 100 "perfectly qualified" panelists and a "ideal" ratings methodology, and lets face it we're far from that, averaging their results means you're going to end up with a Waldorf salad.  Personally, what Rater A thinks is a great golf course may be of little to no interest to me while Rater B may have my interests and ability perfectly at heart, while others may feel the exact opposite.  Averaging the two tells us next to nothing, but yet codifies the results with an instant status.