It might be a fun game but it wouldn't be golf.
Here's a quote from "Golf is my Game" by Bobby Jones,
"Often, too often I think, putting has been referred to as "a game within a game," implying that in some way the putting stroke is, or should be, different from that employed in playing otehr golf shots."
and later,
"There occurs to me one other reason for saying that putting is a game within a game. It makes it much easier to say or to think, "I played very well, but I couldn't putt," or "He's a fine golfer, but he can't putt." I know--I have done it myself--but how can a man play golf well or be a fine golfer if he can't putt?"
Bob Jones said it, and by G*d, I believe it!