Chip
Personally, I prefer to allow the raters to use whatever criteria they for evaluating a course. It all comes down to preferences and we all can justify this, that and the other - the question is to get a decent cross section of panelists which represent all aspects of course evaluation and perspectives. I don't like the idea of assigning a ranking number to a course. I would rather see rankings done in clusters of courses which are more or less equal, a cluster could be 1 or it could 50. I would then like to see the clusters separated by a clear division - say the difference between getting 50 votes compared to 40 votes. Trying to decide ranking numbers of course is crazy.
For instance, I may cluster the following using any criteria I like which basically tends to be where would I be happy to pay full whack for a revisit based on a lot of factors some of which I clearly have nothing to do with golf and focus on the experience (but then I think everybody is influenced by outside factors - its just a matter of who admits as such) and only mentioning those in which I have a decent memory of the course:
A. Enniscrone, Carne, Pennard, St Enodoc & North Berwick
CLEAR DIVIDE
B. Portstewart, Ballybunion, Sandwich, Prestwick, Brora, Donegal, Tenby, Burnham, Portrush, Sligo, Southerndown, Harlech, Aberdovey, Beau Desert, Addington, Woking, Huntercombe, Whittington Heath & Kington.
CLEAR DIVIDE
C. Perranporth, Lahinch, Dornoch, The Island, Saunton East, Machrihanish, Castletown, Southerness, Princes, Dunbar, Deal, Cruden Bay, Aberdeen, Littlestone, Conwy, Co. Down, Portmarnock, TOC, Nairn, Wallasey, Saunton West, Westward Ho!, Gullane #3, Woodhall, Stoneham, Little Aston, Delamere Forest, Notts, Worplesdon, North Hants & Handsworth.
All of these courses are very good to great and I would like to see them all again, but I don't see how I can clearly rank them from 1 to whatever. Does this make sense?
So far as any system that already exists the top100 ranking is far and away the best at least for the UK & Ireland. Of course, this system can't exist without the others so as a stand alone system it can't work.
Ciao