I believe that GD, GW, and other's need these annual ratings to sell paper, that provides income for writers, photographers, etc, down to developers, pros and greens keepers and to advertisers... It doesn't matter what their criteria are... so as several have suggested, why not have a CGA.com criteria set?
I believe there's already criteria on the GCA.com homepage within Ran's descriptive prose.. throw up to a hundred points at each, mix and meld them against each other as you see fit, compile and sort, voila' .. It would be nice if GCA.com could provide a means of capturing such for every course where folks have played and to average their determinations.
1) Enjoyment is the primary theme
2) place in history and improvements/setbacks in golf course architecture.
3) architectural periods divide the courses for comparison
4) minimalism and natural challenge
5) modification needed to adjust to equipment changes.
6) architecturally imposed movement and shaping of land
7) hazards and features of strategic interest
continuity of style
9) brute length and strength required
10) variety within and between holes of shot making options
11) rennaisance value, the king is dead, long live the king!
12) manufacturing quality & visual impact
13) playing strategy risk & reward
14) charm & subtleties in design for play
15) subjective art form versus function
16) the tests of time, as play begets maintenance, maintenance begets continuity, continuity begets understanding and understanding begets a return game.