I think this might be getting a little off the course of the opening post, but educational nonetheless.
Let's be frank, if we are going to be intellectually honest then very few of us will know a golf course initmately. Perhaps we will know one or two courses intimately, but the bulk of these courses we will see once or twice and can make some observations based on those few plays. But to be honest, that is all we will have: basic observations.
AND...if we talk about those observations and others share theirs, then we might be able to piece together a body of knowledge on a course that goes beyond our few isolated plays. In fact, I think that is the idea behind these panels of raters and that entire process. One rater might have played Course X in high winds, another in mild conditions, another in summer, another in fall, etc, etc, etc...compile their individual thoughts and you should end up with something more in-depth in terms of analysis.
In fact, CB MacDonald said "I do not believe any one is qualified to pass on the merits of any one hole, let alone eighteen holes, unless he has played them under all the varying conditions possible---varying winds, rain, heat, frost, etc." There can be no question that he is correct. If we are indeed going to be intellectually honest, we need to accept this fact and simply offer our basic observations regarding courses where we haven't played under enough differing circumstances to make appropriate evaluations.
Anything less than that is hubris.