Paddle is strictly a winter sport, except for a small handful of nuts who play it in the summer. (The guys I know keep a supply of balls in the fridge so the balls don't bounce too much.)
It is a perfect northen compliment to golf (enough out of you southern guys...) except for the late October/November overlap when our league play starts and I'd rather be playing golf.
I have played in 5 degree weather and been fine. You start with many layers and after 20 minutes you have only a layer or two left. I assume the guys in Minnesota are tough enough to play in sub zero weather, but I might take a pass on that. It is an awesome workout when you get four players of similar ability, I come off the courts dripping in sweat no matter how cold it is outside.
The game was invented by tennis guys in Fox Meadow (NY?) who were looking for something to do in the winter.