Eric,
I understand your point and that is one of the issues with all rankings.
So when I voted, I looked at this.
The grading scale is the Michelin three-star system:
*** One of the very best courses in the world, modern or old
** One of the top ten modern courses, if you had played all of those on the ballot
* A very good course, but probably not in the top ten overall
0 Not in the same league with the others on this ballot
Obviously one could look at *** as being a top 10 course in the world or say a Top 100 US course depending on what their interpretation of what "very best courses in the world" means.
For me its the 3 star ones that are the hardest to rate. I used a Top 15-20ish in the world as my barometer when I voted, and so Pacific Dunes is the only one who made it. If I used a top 100 ish in the world as my barometer, then 3 or 4 more of those I had played would have been 3 stars. But since 2 stars was Top 10 modern courses since 1995, and I figured that in a Top 100 list probably would include about 10 or so modern courses, those that I think fall between 20 and 100ish were my 2 star courses. We all did this differently and because when you are only grading on a 4 point scale, the averages change significantly based on the voting. If you were rating on a 100 point scale, the variability would be significantly less...
Don, your points about "How do you know" are spot on but that is the issue with all rankings....