My first suspicion would be that there are distinct obstacles that are out of play for the bogey player from the back tees.
I encountered this myself at Harding Park (though it doesn't actually have a lower slope), where, when playing the back tees, effectively none the fairway bunkers are in range for me off the tee, so I am able to just swing away with a driver without a care in the world. Meanwhile the scratch players are occasionally ending up in them.
From the middle tees, the scratch players should be able clear those same bunkers, when I'm occasionally ending up in them. Thus, comparatively, I'll probably score worse from the back tees, but when the scratch player has do deal with bunkers, they might just score worser.
It's just one factor, and serendipitous I'll actually be reading the rating guide soon, so I should have better information for you, but that is just my suspicion right now.