Matt,
I must go back and take some remedial writing courses. Or perhaps it's my thinking that's gotten foggy.
First of all, I never suggested that raters should be rotated out for freshness. What I said was that there are plenty of good candidates out there who can step in and do a great job when we go. It would be unfortunate if a good rater was weeded out due to seniority because I believe that experience- breadth and depth of knowledge- is the key to the process. As you know, GW management culls out the bad apples, and that should be enough.
My main point, and I apologize for being so repetitive, is that for raters to do the very best job they can, they need to have broad exposure. If we can't see the Pine Valleys, Augusta Nationals, Seminoles, etc. how do we know that Brookside or Brook Hollow are really 7.5s and not 10s?
Likewise, if most raters haven't played Sand Hills or Friar's Head, how are they to determine that a Cog Hill #4 is a 7.5 instead of a 9.5? I would be curious to know how many "fresh" ballots GW gets annually on the three aforementioned classic courses.
As to the many people with the contacts to gain access to all the top courses nationally, who also possess the knowledge and passion for gca, and are willing to be raters, I'll just have to take your word for it. Like I said, I know of just a couple on the GW panel, and both are retired.
We all have our biases or preferences that are brought to the process. I know many raters who wouldn't be caught dead playing the tips. Some can't or won't walk the course. Do you think that they would see Dallas National with a perspective similar to yours? To them it is a long, brutish, unwalkable course. To you it may actually play on the short side with the greens and surrounds providing the course's primary defense.
And yes, concensus, or the cumulative results of large sample sizes, does offset many errors, particularly if the outliers are not normalized. GW could very well have reached a good number with its current panel. I don't have a clue as to what the optimum would be, but I am fairly sure that whatever the number is, it is important that each is rater is well-versed and widely traveled.
I would like to see courses like Dallas National and Friar's Head get wide exposure on a national level. I am not terribly concerned about it, but regionalism could very well be a problem, which could be made worse by having a split panel. My recommendation: encourage all raters to see as much as they can throughout the country. Those who are overly aggressive and discourteous, deal with them harshly but fairly. For feedback and learning, I would also like to have the raw data that results in the list available (without specific rater identification).