I think that "polarizing" is better left to politics and music. Honestly, don't all courses consist of sand, grass and water? of trees, bushes and weeds? of tees, fairways and greens? On my watch, you like a course mostly because you played well, because you and your group play there, because access is logical.
You get to know a course and can comment on its value after numerous playings. If you don't like the course, you're not likely to return. Therefore, dislike of a course usually comes after a single trip, not after numerous rounds. If you return a few more times, you're likely to find some things to like and the course becomes less despicable, less offensive.
I agree that Tobacco Road can be a frustrating course for those who do not play to their strengths. The carry up the slope on 9 or the carry over the quarry on 18 are quite intimidating. If someone is frustrated after holes one and two, hits the green on three in the wrong spot and four putts, the remaining 15 holes are meaningless...terrific but meaningless.
I have recommended to Mac Plumart and any others who will listen that they should play True Blue before Caledonia and Tot Hill before Tobacco Road. Should you play Royal New Kent or Stonehouse first, when in Williamsburg?
I would think that the element of the drive is the most important when defining the potential threat of non-appeal. The drive defines the majority of golfers, not the approach, not the putting. Drive the ball well and you'll usually slop it up near the green and save pars and bogies. A golf course that allows ample width of fairway and allows the golfer to get the ball close enough to the green will never disappoint. A good putting day is unintentional; most golfers work on their putting so infrequently that they need to be "on" in order to putt well; the touch soon reverts to the average, 34-40 putts per round.
In conclusion, I'll give my two cents on what might be considered a "polarizing" course: Southern Pines. All of 6200 yards, many of its drives go into an uphill slope, reducing role. The need to add a club or two to the approach, without even factoring in the wind, is beyond the ken of most golfers. Ergo, approach shots end up short. The greens are diabolical and speedy and the three-putt is no guarantee for the average Giuseppe. In contrast, the architectural aficionado loves the joint. Go figure.