All in all the handicap system works pretty well.
You cannot relate a handicap system to many sports. The example given of bowling I can understand. It is an individual sport where numbers are used to calcualte a final score. Those numbers can be analized and a specific handicap can be given to anyone. The example of hoops does not have a chance. You would end up spending four or five games coming up with what the "starting spread" should be and I believe even after that the better team would still win most of the time if they stepped it up at all. Not fun for anyone to play to 15 and start at 14-0.
The pro thing is an interesting issue. Typically, most professionals play off of scratch, but if that pro is better than a scratch most memberships will request that the pro play off a handicap that is lower than scratch. My membership forces me to keep a handicap and I'm okay with that. The system works and it is fair to all concerned. If you play to your ability and your opponent does the same, it will make for a great match. Whether it is a five against a twenty-five the system works. You should here the discussions when a pro plays against another pro and there needs to be shots involved. The end result is that everyone should get what is coming to them professional or not. I'd love to take my 6 shots from Tiger with his plus 8 handicap.
When I play against most "honest" (meaning they put in all of their scores) amateurs, the fact that I keep in my back pocket is that most player's handicaps are usually slightly lower than they should be. If I simply make them to play by the proper rules of golf I usually end up getting an extra shot or two. How many times does a player hit a ball into the trees and say "oh, we'll find that" and end up not finding it and simply dropping a ball beside the trees and play it as a lateral or with no penalty at all. There is also the intimidation factor that I can usually count on for the first few holes to help me out. If all fails and I am losing on the back nine, I can always resort to a couple of bad swing tips to throw my opponent for a loop. (Just kidding, I would never such a thing)
The reasons for the lack of improvement over the years have been mentioned by many others on this thread, but the biggest reason is the revolving door of players. There is always roughly the same percentage of new, aging, core, casual and many other categories of golfers that are entering the game and leaving the game. It is this total group that form the average handicap. One other big reason for the lack of improvement is the vast amount of golf information that any golfer is subjected to. Too much information is not always a good thing. Thank you Golf Channel, countless magazines, golf schools, infomercial gadgets and who knows how many more. All of this keeps increasing the interested in the game, but thankfully helps to keep all of us golf professionals busy trying to sort out all of this confusion that ends up in someone's head.
Golf is a wonderful game........