A little anecdote I've relayed to a few people on this topic, back when I was in college I and two of my friends used to play regularly with an elderly doctor in the club's Wednesday fourball, he was a great guy, loved the game, a student of the game but also used to buy all the latest equipment as soon as it came out. He used to say to us that he loved playing with us as we would hit it so far past him, he ignored us and would play his own game, however when he played his normal Monday and Friday game and one of his friends would hit it past him that normally wouldn't, he would be jumping all over the next tee shot to make sure he'd be longer next and many times this would lead to worse results than he would have liked.
The moral of the story is distance is judged by comparison not by the actual distance, is 180 metres short? Is 300 metres long? 180 metres is long compared to 10 metres, 300 metres is short compared to 3 kms.
Golfers compare the distance they hit to other golfers and in particular to the golfers they play with the most, with a rollback where everyone hits the ball shorter than before won't bother golfers as much as others think, as they will still just compare themselves to the golfers they play with.
The simple argument for a rollback is sustainability, a shorter ball means shorter courses, less area needed, less inputs needed, less resources needed, better to voluntarily use less resources rather than forced into it at a later stage and not be prepared for it.
The nature of the game argument is a poor one, as different golfers have always played holes different and will always do, even golfers at the highest level don't play the same shot, same club in the same way to same hole all the time, the hole can even play differently from the morning to the afternoon.