I could definitely see this being the case on some short par 4's, though I'm not sure I'd say it's universal. #10 at Riviera would be harder downwind I would think, but a hole like #3 at Dormie would probably be easier downwind?
With that said, #10 at Riv runs almost due south, so I would think the wind would usually be crossing.