The original intent of USGA research was to find a method to build greens using local sands to build a practical, affordable green.
As they did more research, people started adhering to the "recommendations" ever more closely. Then, with more research, they get tweaked some more. Now, as mentioned, everyone seems to be afraid to vary at all from USGA specs, although Mike Hurdzan and a few others have led us to alternates (like California)
In 1968, when USGA greens debuted, irrigation and fertilizers had progressed far enough to be able to replace those agronomic needs, so sand made the most sense as a base because compaction was the biggest issue left. It still does, even when you don't follow recommendations to a tee.
Quite simply, sand compacts less than soil (originally they thought it wouldn't compact at all, eliminating the need to aerify, although that hasn't panned out) and compaction is literally a green killer. Most soils compact too much under the foot traffic that greens get, and short turf would be nearly impossible. (key word, nearly, esp. on heavy play courses)
I always use sand based green, but look for performance characteristics, as originally intended, and try to use local sands and/or California greens to keep costs down, rather than always follow USGA specs.
I have seen the USGA research, and lo and behold, the gravel layer actually does speed drainage and even it out, vs. pure sand greens, which is always nice, but it is expensive.
Another issue is "calcarious" sands which most agronomists don't recommend. However, the biggest issue is those kinds of sands can tie up fertilizers. Not good, but we have to balance that out against initial costs.