Anthony,
Your basic question revolves around a system of course rankings that began in the late 1970's. Why is this important? It isn't as if there were annual magazine course rankings since the teens, with Pine Valley "Breaking Into the TOP TEN" at its opening; no it is a system that was devised and applied by the relatively modern golfer and is still applied by today's "golf experts."
The real question then should be WHY do modern players believe that the great courses of the "Golden Age" top have been far and away better than contemporary designs?
I think the answer lay in the subtlety of the system of the Golf Clubs that we have developed since the game was first introduced in America. Consider, what is the single greatest advantage AND difference that the Golden Age clubs had over today's? The FOUNDING MEMBERSHIPS!
When the original 1890s Baltusrol course was being replaced in 1918-22 with two new ones, Tilly was given an unlimited budget and left alone. The same thing for Winged Foot right after that and a number of his other great courses. Look at those who entrusted CB Macdonald to simply create the best course possible in hopes that in EXPECTATION that NGLA would be the greatest course in the world; not America but the WORLD.
The Amateur architects had the same lofty goals and also the same ability economically to attempt the same challenge. Crump at Pine Valley, Fownes at Oakmont, the men of Merion; all trusted and expected those that their courses would be recognized as THE course.
Today's new golf club founding memberships that desire and can afford the same potential quality in course design expect the same thing, but it is in the process of bringing it about that the differences lie, for a Tillinghast, a Ross, a Mackenzie, had a greater freedom to create then than a Doak, a Coore & Crenshaw, and a Mike Young do today.
Today's great architects are hampered by their owners desires as crafted by their personal visions of what that should look like. The results being outstanding golf courses that lack that last spark of... transcendent inspirational genius. Look at the difference in how two courses have come into being to highlight what I mean. Both clubs were built with the idea that the land for the overall project would be used for two different things; golf course and housing.
Longshadow Golf Club... Mike Young will tell you that his choice of routing and hole locations were limited by the needs of future home sites; in other words, the housing parcels came first.
Merion Golf Club... Here we have the same thing except that the land to be used for the golf club was given PREFERENCE over that for the future house plots. Better land ALWAYS produces a better quality course.
That is just one of the modern interfering factors that the great golf architects of today face that those in the "Golden Age" did not. It is those differences that have brought about an ability for exclusivity of use that created a natural course ranking system before anything formal was put on paper, for otherwise why is ACCESS such an issue for those who desire to play exclusive clubs?
Our desire to enjoy what we are not allowed to simply because of who and where we are instills a subconscious recognition in us that the object of our desire MUST be better. Be honest with yourself and make a list of the top 25 courses that you would love to play. What is the reality that you will actually be able to do so? Yet when that rarest of privileges happen and you find yourself walking the rolling seaside links of Shinnecock or within the quiet trees of Pine Valley or enter the grounds of the LACC, do you honestly think that the course won't meet up with the mental expectations to which you have imbued them?
High expectations lead to the greatest disappointments, but more often they also lead to the strongest affirmations of what was already believed.
Despite all of that, Anthony, take a careful look at the list of the top 20 golf courses in America and the Top 30 in the world. Consider how many courses are of a more recent vintage than the greatest of the ODG's. In fact, isn't there a course in Las Vegas that was built by a casino owner whose desire and funding to see the best course in the world built on his property give free reign to an architect whose creation would DEBUT in Golf Digest's Top Ten?
To me, within desire, funding and freedom lay the answer to your question...