I am as much as, if not more of, a strict grammarian and usage snob than most, but I find this debate a bit too semantic--and "gotcha"--even for my tastes.
Both literally and especially figuratively, Johnny was correct. He was, dare I say, using the vernacular of the English language.
Most of this thread focuses on the principal literal meaning of "unfair," but there are multiple, albeit closely related, literal meanings. We are all focused on the most common meaning, which deals with inequality. But there is a second well-known meaning, which deals with justice, and it accords with the point Johnny was trying to make, and made. Even if something affects everyone equally (i.e., taking us out of the principal meaning), it can still be unjust (i.e., putting us into the second meaning) in that it fails to accord with our reasonable expectations of what is just and right (or, as the dictionaries put it, is "disproportionate; undue; beyond what is proper or fitting"). “Too easy” or “too hard,” in other words, CAN be “unfair.”
But even if we ignore this other literal meaning and assume Johnny was literally incorrect, no one can reasonably dispute that his word choice accurately conveyed to the viewers what they were seeing with their eyes. He is speaking to a national audience on a cable network; he is not speaking to golf junkies on Golf Channel, and he is most certainly not speaking to golf-course-architecture aficionados on Golf Club Atlas. If the opposition to his word choice is precisely because he's speaking to a national audience--i.e., especially to those not in the know and to those most likely to play or not play the game based on what they see and hear on TV, suggesting that golf is in any way "unfair" deters possible participation and stunts the game's growth--I think that opposition is, at best, far-fetched and misplaced.
[UPDATE: Jeff's post, which went live at the same time as mine, reflects where the focus of this debate should be--i.e., on the playability of the hole, not the adjectives used to describe it.]