Ally,
It's a great question. I've played over 800 courses and up until a few years ago (I'm 52), I could visually recollect about 95% of the holes I'd played.
I would do some memorization drills that helped. After each round, I'd first try to go through all of the holes again in my head with no outside assistance. Then, I'd do it again with the scorecard, or course map, and finally, I'd do it a third time by recalling the holes in Handicap order.
It helped fill time on some long drives, and yes, I did try to keep both hands on the wheel.
Evidently, whatever area of the brain all that got stored in has been purged in recent years, however. These days, I'm probably able to recollect about 40% of all the holes I've ever played.
I'm hoping that I simply now have more space to bring in new courses and holes, but it's likely that my brain facility has simply shrunk over time.