You're on the right track with moving to a database Paul. But the best long term scaleable solution is for an organization to copy the Amazon or Travelocity model and create an algorithm.
This algorithm would provide a mathematical solution to acheive the flexibility needed to benefit the user. This is achieved by weighting the inputs of all users differently depending on their skill and experience levels on multiple interlocking scales. And by providing results that can be individually tailored to the desires of the user.
Examples:
A) My fathers ratings of Cincinnati golf courses on conditioning vs value would be heavily considered, because he's been playing and closely comparing them a couple of times a week for 50 years. But his ratings of Camargo vs Pebble Beach vs Carnoustie in most categories would be heavily discounted, because he's only played Pebble and Carnoustie once.
B) My wife scores for most evaluation categories would be heavily discounted, because she's relatively new to the game. But they'd still count things like aesthetics or walkability, because anyone can have an opinion on whether a place is enjoyable to spend 4 hours walking around. And she's qualified to opine on which of Bermuda's courses are the prettiest walks.
C) Ben Crenshaw's ratings of the top 1,000 golf courses would be heavily considered against each other in categories like strategy, difficulty etc. He'd be a "prime link". But probably pretty discounted in value, or for a course in Louisiana that he hasn't visited in 30 years.
The algorithm identifies and eventually completely ignores the scoring of someone who's continually entering exaggerated positive or negative scores from a course or courses. The same as travelocity ignoring the scores from the 3rd shift manager who's punching in positive reviews for his own hotel every night. Similarly, the algorithm can increase or decrease the weight of specific scores depending on their variance from a trend line. This can be done for the courses and from the raters own trends. This quiets the effects of a bad weather or "played great" day.
And the algorithm can continually rebalance the scores of a course AND of a rater as the rater gains more experience.
My understanding is there is a group that's already in the process of trying to raise capital for a for-profit website/app that would follow this model. I think it has potential.
BUT, isn't one of the simplest short term solutions for some of the existing ratings providers to just make their rankings more easily sortable by the users personal priorities? Tom Doak has already done this in his scores - he's plainly stated up front that his scores are meant to be used as a guide for helping you choose where to play while traveling. For the bigger media outlets, why not release the data by category for every course and let the user decide how to weight each category? Seems simple?
Michael