Jeff,
Great thread. I have to say I agree with you on most points. Who cares what the pros think about a golf course? I have started to ask myself this question more recently. They do represent a small percentage of golfers, though they represent a larger percentage of the total rounds played. Nevertheless, it is central to the golf equipment debate. Why do we need to lengthen courses to accommodate the 1% of golfers that benefit from equipment advances? Why are golf courses considered obsolete because they are too short for a small percentage of golfers?
I think you are correct that should be more of a focus on fun courses in the game. I think it is fun to play a tough course. I would love to play Pine Valley a couple of times of year. But it is a course I could play every day? Is it course where one can learn the game? No and No. Yet many courses are built in the Pine Valley mold. They are inaccessible and no fun for most golfers.
At the same time, there is still a mass appeal to long and difficult courses. People love to say they have played at a US Open course. People care when other people play at Open courses. When I go back to the states, most of my friends will not care that I played at Huntercombe or Swinley Forest, as much as I might. However, they will be interested to hear that I played at next year's British Open venue. Is Swinley Forest or Huntercombe a better course for the general public than Sandwich? Probably. But the general public will flock to Sandwich over Swinley because Sandwich is the course that can beat up the pros. Simple as that.
Would the top pros change? Some would, some would not. Tiger would still be the best player in the world. The courses you listed put the emphasis on short game over long game. He has an imagination around the greens that is only matched by Mickelson. These two would still be the best in the world.
What this paradigm shift would do is make the pro game more competitive. It would bring more players into the fold. Shorter courses will allow longer and shorter hitters to compete an equal spectrum. I think this could only grow the game.
For your biggest question, I think golf would grow and would be better off if courses like Brora and Painswick were rated higher. At the same time, I really do not care where these courses are ranked. Courses are made for golf. They should not be made to compete for a spot on some superficial list.