How I rated Yale using Golf Digest's criteria: request for help

[Mark Bourgeois]

My New Year's Resolution is to live the Golf Digest rankings system. Step out of my preconceptions and comfort zone and try to learn something new. Not just knee-jerk oppose the system but give it a chance.

This evening I took my first shot at actually rating a course: Yale.

I scored it 58.6667, a score that, were it to reflect the entire panel's view, would make it the 161st-greatest course in America, just a shade greater than Whistling Straits (Irish).

I did my best to do it the right way. Hopefully, that's apparent in the score -- personally, I consider the course one of the 30 greatest in the world, probably top 20.

But I don't think I did things right and could use some guidance, especially from the GD raters out there.

First problem. Originally I scored it several points lower, knowing Golf Digest panelists are not particularly favorable towards the course. I wasn't trying to make a point, I was just trying to be a good rater and do a professional job of it. I thought I scored it properly. But then I "peeked:" I took a look at the official number. And after seeing I had it "too low," I went back and bumped it up. I'm not particularly proud of doing that, I just figured maybe I didn't understand the criteria fully -- I don't -- and tried to calibrate accordingly.

Does anyone have any advice for how I can do this better? I assume Ron Whitten tells us we shouldn't look at the official score but let's be realistic: in this day and age it's impossible NOT to know what others think and impossible NOT to look at the scores. Who the hell rates for a magazine but doesn't read the magazine's ratings?!

More struggles:

I found myself fighting to avoid just "lock-stepping" Shot Value scores and Resistance to Scoring scores. After some thought I was able to see how a hole could score highly on RS but low on SV: just make it a million miles long and boring. But I'm having trouble seeing how a hole can score highly on SV but low on RS. How is that possible? Can someone share example holes that "pose risks and rewards and equally test length, accuracy and finesse" (SV definition) but are easy?

Conditioning: Yale is a course that, despite an excellent greenkeeper (he won Superintendent of the Year a few years ago), more or less is at the mercy of nature. I know to a degree all courses are but Yale really is. I'm not very good at paying attention to conditioning so I knew this would be a problematic category before I started but I ran into a very specific problem.

I've played the course in various states of conditioning. And if I were a really good golfer, maybe I even played it in an NCAA tournament, where conditioning I'm told has been above average to very good. Should I rate it for peak time of year, worst I've seen during the rating period, best, average, what??

One last point relating to my conditioning conundrum: I am glad I don't have to rate a links for some of these criteria. In particular, I have no idea how anyone could begin to sort out SV and RS. Has anyone tried??

Thanks for any help.

[John Kavanaugh]

If you give your individual scores for each category then I believe I can help. Please also list list your 1st, 25th, 50 and 100th best courses in the world so we have some basis to see why you think Yale should be 20th.

I personally think you nailed it at 161st. That is a bit higher than any other course with a super of the year.

[Mark Bourgeois]

Thanks for the offer. That top 20 stuff is just my own personal opinion and therefore without merit. It's not like I use an authoritative, tried-and-true criteria-based system. The GD number is the authoritative one I'm going to have to get used to.

My scores -- keeping in mind I was trying to imagine things as a scratch golfer and not my own game or perhaps even knowledge about the course:

SV 7.0000
RS 6.6667
DV 7
Memorability 10
Aesthetics 8
Conditioning 5
Ambiance 8

Well...can you help me?

[Mark Saltzman]

A 7 for DV? Seems low.  Bump that a point and get the course after a few dry days and all of a sudden you're in top-100 Greatest territory.

Also, RS 6.67? Length alone does not determine RS.  I think the 10th hole scores a 12/10 

[John Kavanaugh]

Thanks for the offer. That top 20 stuff is just my own personal opinion and therefore without merit. It's not like I use an authoritative, tried-and-true criteria-based system. The GD number is the authoritative one I'm going to have to get used to.

My scores -- keeping in mind I was trying to imagine things as a scratch golfer and not my own game or perhaps even knowledge about the course:

SV 7.0000
RS 6.6667
DV 7
Memorability 10
Aesthetics 8
Conditioning 5
Ambiance 8

Well...can you help me?

I can't help you until you tell me what you think instead of what you think other Digest raters would think. You don't believe that the shot values on a classic Raynor is only at 7.0. Do you understand the difference between a scratch golfer and a competitive golfer?  6.6 Resistance to scoring?  That's not your opinion. Conditioning at a 5?  That is another lie. Why are you afraid to give your opinions?  Do you think Whitten will kick you off the panel if you don't pass his personal rater litmus test?  That crap only happens at Golfweek?

The beauty of a panel with 800 raters is that the guys who love Raynor balance out with those who don't. If you are not true to yourself you are wasting everyone's time just to steal a bit of golf and live some fantasy power trip. Now tell us what you think not what you think you need to say to keep your card.

[Mark Bourgeois]

I have received a personal promise my rater card will never be stripped from me.

I'm trying to be true to the criteria, to execute a score reflecting a course's standing against that criterion. Like conditioning: I don't care. Obviously I can't put that score down so I figured sometimes when I play it its in great shape, sometimes, like after a storm or during a drought or a heat wave, the course struggles.

Good days...bad days...eh, average. 5.

How would you tackle it?

Re SV the driver has had a huge impact. I played the course regularly up to 1999, then not at all, nada, from then until summer 2012. I could not believe how much shorter clubs I held in my hands for approaches. But it's interesting in a RS kinda way: used to be the teeth was in the tee to green. Now it's the greens that frustrate golfers. Not a lot of one putts out there.

Another question is, after seeing the SV and RS scores, did I rate the other categories on the merits or inflate them to get the course closer to where I feel it belongs? I tried not to do that but was aware there could be subconscious lobbying. How do you solve for that?!

[Sean_A]

Mark

Considering conditioning, and I don't know what Yale's policies are, do you think courses that stay open all year will or should suffer in this category VS courses not open all year and liable to shut down when bad weather hits rather than stick a few temps on etc? 

Sorry I can't be of any help with the Yale rating.  If it helps in an indirect way, I did TOC using DG criteria.  It came out to 61 something.  That places TOC just above Hudson National - somewhere around 90 in the US.  I did this without the Bourgeois peak/adjustment (which I think GD raters do all the time to make sure courses aren't out of whack to their "feelings").  I then realized for my DG ranking to mean anything for ME, I would have to rank many courses to know where TOC fits in the system.  I could be that TOC is #1 with 61+ average.  What it more likely means is that I am far tougher rater than most on the GD panel. 

Ciao

[Patrick_Mucci]

Sean,

That's a good point about conditioning.
Time of year and/or Mother Nature can have a huge influence on that category.

Mark Saltzman,

You indicated that length alone doesn't determine "resistance to scoring", would you concede that it's the "primary" or "majority" of the factors that constitute resistance to scoring ?

I have to believe that other than a hole with quirk, that distance comprises 90+ % of resistance to scoring.
I would offer the "Redan" as exhibit "A".
At 190 yards that's one hard hole.  At 150 and less, not so much so, yet the architecture remains unique.

I can't think of many short holes that truly resist scoring.
# 1 at NGLA comes to mind, but, I can't think of many others off the top of my head.

Mark,

I think JakaB has a valid point.
You're looking for peer approval, not candid assessment.
You're looking to discover the process by which you can fit in with the herd and that's one of the problems I have with the ratings, no one wants to be an outlier.   Most are predisposed and you're trying to "fit in" rather than provide an assessment based on the given criteria.

[Mark Bourgeois]

Sean:

I don't know how I'm supposed to treat this category, that's why I'm asking. I assume for most raters it's not an issue because they only play the course once. Maybe they sort of guess at how it plays the rest of the year? Or maybe they adjust their score based on what members and the pro shop tell them. You know, "Oh, you should have been here last week" stuff.

You also reflect a solid point made by John K: do I need to calibrate against myself by rating other courses? But if I do and like you find myself a tough grader, then what do I do, create some sort of Gormless Multiplier to apply to my scores?

Mark:

All the par 4s are between 365 and 440. That doesn't sound like DV to me.
Over the years I have seen plenty of players par 10, no big deal. I par it a reasonable amount of time given my abilities.

At the end of the day for really good golfers it's a drive to a wide fairway followed by PW-9I. Other than two blind shots, which better players don't even notice, its defense is its green. Not too much trouble for a scratch player.

It's not a pushover but once familiarity sets in it's not that brutal. I think I scored it an 8 on RS, maybe a 7, I can't remember.

Back in the 1990s before magazines like Golf Digest cheered on equipment changes it would have scored a 9 or 10 in my rater scorecard.

All:

I'm not afraid of being an outlier because of my beliefs. I am however "afraid" of being an outlier because I misunderstood the criteria. That's all. Frankly I find it hard how any rater could be an outlier for SV and RS except by misinterpretation. There still could be variation in these scores due to the differences in average drives hit by scratch golfers.

I do see how the other categories could yield groupthink because they're assessed at the course level and are more personal, less objectively measurable.

[Patrick_Mucci]

Mark,

Don't you think it's incumbent upon those crafting the rating system to make sure that you understand the system PRIOR to sending you out into the field ?  ?  ?

If the methodology is so esoteric that it defies understanding, wouldn't one have to question it's validity ?

[Mark Bourgeois]

Patrick,

I'm not a Golf Digest rater, I'm just doing my best here. Yes, of course those crafting the system would train me up before sending me out. Rater handbooks, three-day weekend sessions, etc. Right?

That's why I went back to calibrate my score. I don't know if I'm doing this right. That's why I'm asking for help.

You don't believe that the shot values on a classic Raynor is only at 7.0. Do you understand the difference between a scratch golfer and a competitive golfer?  6.6 Resistance to scoring?  That's not your opinion. Conditioning at a 5?  That is another lie. Why are you afraid to give your opinions?  Do you think Whitten will kick you off the panel if you don't pass his personal rater litmus test?  That crap only happens at Golfweek?

The beauty of a panel with 800 raters is that the guys who love Raynor balance out with those who don't. If you are not true to yourself you are wasting everyone's time just to steal a bit of golf and live some fantasy power trip. Now tell us what you think not what you think you need to say to keep your card.

Here's how I scored each hole on SV and RS:

Hole   SV  RS
1        8    7
2        7    7
3        7    7
4       10  10
5        5    4
6        8    5
7        5    5
8        8    7
9       10   9
10       7   8
11       4   4
12       7   7
13       8   7
14       7   6
15       6   6
16       3   3
17       8   8
18       8  10
Avg     7   6.6667

[Phil McDade]

Mark:

Acquainting design variety on Yale's par 4s solely with yardage ("All the par 4s are between 365 and 440. That doesn't sound like DV to me.") suggests you've fallen into the trap commonly associated with GD raters in that they view yardage primarily, and sometimes soley, in relation to factors such as variety and resistance to scoring.

One question I'm curious about: Do you think categories such as shot values, variety and esp. resistance to scoring are primarily focused on tee-to-green play, and that those factors are then "weighted" less once a ball gets on the green? In other words, do you think -- or did you find yourself -- viewing a 5-iron over a penal hazard differently than a roller-coaster putt of 40+ feet over two or three contours in a green? My sense is that GD raters tend to weight the tee-to-green portion of the round more heavily then the putting part of the round, when in reality just as many strokes are taken on the green as other portions of the course. (Example: What's a tougher shot -- the tee shot to a front pin at Yale's Biarritz, or the subsequent putt from the back third of that green to the hole?)

[jeffwarne]

Mark,

Don't you think it's incumbent upon those crafting the rating system to make sure that you understand the system PRIOR to sending you out into the field ?  ?  ?

If the methodology is so esoteric that it defies understanding, wouldn't one have to question it's validity ?

I wonder which category box the GD rater was looking to tick when he repeatedly asked me how much money we spent on azaleas?

ambiance(create a certain mood)
conditioning (they're certainly beautiful and would be "maintained")
resistance to scoring(very penal)
Memorability (see ambiance and conditioning)
Shot values (sure you'd consider them on certain shots)
Aesthetics (see above)
DV (don't even know what that is but given how similar the rest of the category boxes are ,....see above

The "azaleas" he was referring to were mountain laurel, which is abundantly native to the site, and NONE was planted, although a lot was pruned.

[Josh Tarble]

Mark,

I think John and Sean have made a good point in that you need to try to rate a few other courses in the same manner as you are doing with Yale.  I would think you should try to rate one you are more familiar with than Yale, one you are about the same familiarity and one you are less familiar.  This should help establish a control group for how you rate.

[Mark Bourgeois]

Phil,

Well, my own thinking -- trying to put myself into the head of a scratch golfer -- is that the primary challenge these days is hitting the correct section of Yale's big greens. If you do that, you've got a two putt and possibly a realistic birdie.

If you DON'T hit the correct section, then birdie's out of play (unless it's the 16th, in which case eagle's out of play) but par isn't lost but you've got some work to do. This is the only significant place where I'd expect to see scratch players losing strokes: three putts due to putting from the wrong section.

But as we know, one thing that links scratch golfers together, certainly more than driving distance, is putting, so I'm not sure how often 3-putts occur.

In sum, I think there are only two primary types of "shot values" out there for the scratch golfer: hitting the correct section of the green -- loss of birdie, possible 3-putt is the penalty / shot value -- and first putts.

I think technology has devalued the shot values of most (driver) tee shots out there.

Josh:

I will try to rate a few other courses and see what happens.

Mark

[Andy Troeger]

Mark,
I think you picked a tough course to start with. Yale has one of my biggest gaps between how I rated it using my "Andy scale" and how the numbers came out on the Digest scale. I love the place, but my numbers were lower than I would have expected.

That said, my numbers in most categories are higher than yours. We're also supposed to give out a "10" in a category about as often as we make a hole-in-one so if you think its that great then its probably more of a 9.5. I do think the course has great variety, however, and would give that a higher score. Conditioning when I was there also was better than a 5, but not at the level of some other greats. All that might come to down to differing views, however. You're pretty close to the course's mean, so I'd say you have the idea.

Phil,
You might be right about Digest raters valuing tee-to-green. My biggest difference of opinion from many on here usually comes with courses that I think are a bit lacking tee-to-green but do have fun greens. GCA posters often still love those courses, where I'm usually at least one step lower.

[Anthony_Nysse]

If you give your individual scores for each category then I believe I can help. Please also list list your 1st, 25th, 50 and 100th best courses in the world so we have some basis to see why you think Yale should be 20th.

I personally think you nailed it at 161st. That is a bit higher than any other course with a super of the year.

Some Past winners of Superintendent of the Year
Sam MacKenzie, Olympia Fields (Ill.) Country Club (2008) Golf Digest #62
John Zimmers, Oakmont (Pa.) Country Club (2007) Golf Digest #4

I wound think these are higher than #161.

[Mark Bourgeois]

Well, Andy, both those holes have appeared in lists of 18 greatest holes in the world, so I'd say 10s are justified. Whoops, see what I did there?

I confess I don't understand shot values on 9 very well. It's just a hard but breathtaking shot that's all carry over water. Take away the history and the views and from tee to green it seems to me like a 1 or a 0.

Anthony, if it were up to me I'd give conditioning at Yale an 8. I think Ramsay has done an amazing job and when the course is on it's on. But I am just trying to see it like a scratch handicapper Golf Digest rater, and I think a 5 is the best to hope for. For that kind of player I think Yale needs a cool, dry (but not too dry) summer. Meanwhile, the rater who plays it in the spring will probably get a muddy-ish course, and the one who plays it in the fall will have to deal with leaves interfering with his play.

Also, the fairways are not iron-smooth so water can collect in the little hollows. You might get a "perfect" lie on a little hummock but then a mushy lie in a hollow. Thus, a scratch Golf Digest rater has no choice but to ding the conditioning score for imperfect conditions.

[Andy Troeger]

Mark,
Sorry I wasn't clear, I meant a 10 overall (for memorability). A 10 on a specific hole is no biggie, because it gets averaged out by the others. But saying an entire course is perfect for a category is unusual, if only because you are saying there's no way the course could be more memorable than it is.

[John Kavanaugh]

Also, the fairways are not iron-smooth so water can collect in the little hollows. You might get a "perfect" lie on a little hummock but then a mushy lie in a hollow. Thus, a scratch Golf Digest rater has no choice but to ding the conditioning score for imperfect conditions.

Can we stop the charade that you are attempting to learn when this is just another thread attempting to prove that Digest is flawed.  I am sick of my new role defending raters but nobody is as stupid as you are making them out to be.  It is not constructive or interesting.

Your premise is that Yale is a top 20 course in the world based on repeated plays during a time that even Dr. Childs had his panties in a bunch over the ignorant management of the place.  In 1999 Snake Plissken would not have paid to play the course on a day pass.

If you really want to think like a Digest rater then look where you want your course ranked and reverse engineer the numbers to fit.  It is kind of like making a cheat sheet back in school in that it does force you to think to the point where you may end up learning something.  It works.  Hell, you might even discover that what you thought was the 20th best course in the world really is 161st in America.

[Greg Taylor]

You and all the raters need to be calibrated correctly....!

If you're going to rate peer-to-peer, then each rater must all understand what is 1,2,3,4, etc... for each category. Otherwise there will be bias based on the differing start points.

Also, the scoring is subjective in certain (all?) respects so isolate the objective elements for a more fact based discussion.... "ceteris paribus".

Ultimately however it does boil down to, "who is sexier Halle Berry or Claudia Schiffer".... it's your word versus mine...!

[Patrick_Mucci]

Patrick,

I'm not a Golf Digest rater, I'm just doing my best here. Yes, of course those crafting the system would train me up before sending me out. Rater handbooks, three-day weekend sessions, etc. Right?

That's why I went back to calibrate my score. I don't know if I'm doing this right. That's why I'm asking for help.

Mark,

I understand, but, you're questioning if you got it right, based upon reviews/ratings by your peers.
In other words, you want to fall within the accepted range rather than your own independent assessment, not predicated upon the prior assessment of others.

If doing it right leads everybody to the same conclusion, why would they need such a large panel.

Go with the FORCE Luke  

[Peter Pallotta]

Mark - I don't know anything, but my guess is that, if everyone wasn't 'peeking' and, indeed, constantly peeking, there would be no consistency whatsoever in the rankings year over year, let alone the overwhelmingly consistent rankings we tend to see. In short, if you want to do this right, I believe that your peek should become second nature - a steady an unbroken gaze as it were.

Peter

[Mark Bourgeois]

Also, the fairways are not iron-smooth so water can collect in the little hollows. You might get a "perfect" lie on a little hummock but then a mushy lie in a hollow. Thus, a scratch Golf Digest rater has no choice but to ding the conditioning score for imperfect conditions.

Can we stop the charade that you are attempting to learn when this is just another thread attempting to prove that Digest is flawed.  I am sick of my new role defending raters but nobody is as stupid as you are making them out to be.  It is not constructive or interesting.

Your premise is that Yale is a top 20 course in the world based on repeated plays during a time that even Dr. Childs had his panties in a bunch over the ignorant management of the place.  In 1999 Snake Plissken would not have paid to play the course on a day pass.

If you really want to think like a Digest rater then look where you want your course ranked and reverse engineer the numbers to fit.  It is kind of like making a cheat sheet back in school in that it does force you to think to the point where you may end up learning something.  It works.  Hell, you might even discover that what you thought was the 20th best course in the world really is 161st in America.

Clearly you're smart enough not to get involved in a land war in Asia or go against a Sicilian when death is on the line, but let me see if I've got this straight.

You're saying if I want to undermine GD's credibility, I shouldn't do what I did, which was rate lower than but roughly in line with GD, but instead give it really high numbers -- and then after I do that, somehow walk the number back down to where I had it originally? Huh?

Anyway, now that Andy's explained the meaning of 10s, I reconsidered my Memorability rating -- now an 8. New score is 56.6667.

Mark Saltzman: I still think my RS score is right -- not too low. Scratch golfers can shoot or beat their handicaps first time out.

[John Kavanaugh]

Mark,

You have no clue what a scratch golfer or more specifically a Golf Digest rater skill level is. Of the approximate 800 Digest raters less than 100 would even break 80 on their first time out at Yale when playing from the back tees. That number is only that high because of the number of + handicaps on the panel. Scratch golfers shoot their handicaps far less often than other golfers. When is the last time you saw a scratch win a member guest?

Was is your handicap or are you one of those guys who don't post scores?

What is funny is that you have so little respect for Raynor that you say scratch golfers can shoot or beat their handicap on the first time out on what you consider the 20th best course in the world. Any course where that is true would be strategically impotent.