Kyle,
In your eyes, A sport requires some level of physical action or exertion and a game requires some level of rules and structure?
So hitting a golf ball is a sporting activity, but once you perform that activity on a golf course with tee markers and flagsticks you are now playing a game?
Where does it end? Is ditch digging a sport, how about sex, how about an overweight man walking from their car to the office on a hot day? If the overweight man is asked to walk between his car and the office in less than 2 minutes does that change it to a game? what if the ditch digger was forced to only use one type of shovel?
You seem to have a grand and broad view of sport, while at the same time a rather limited and confined view of game. You're afraid to define sport and game as that would then reduce your ideal of sport to something less than. Why is the existence of the game of golf harmful to you? Why do you so badly need it to be as much as sport as possible?
Finally, why do so many activities need to fall into either category? Especially outdoor activities such as the ones you've mentioned such as kayaking, hunting, or hiking. I've been a backpacker for more than 20 years, hiked all over the US, and it would have never crossed my mind to categorize hiking as either a sport or a game, it's a pursuit of something different than either.
Much of this discussion has been a "bit of sport" where does that English phrase fall into your definition?