San Clemente and Santa Anita are good "if you're in the area" stops. How far I'd drive to play either one depends on the company. Of course, I've played Santa Anita probably more than any other course in Southern California, or maybe anywhere else, as it was two miles from where I grew up, and the home course for my high school and three other high schools in our league. Great ocean views at SC. Oak Quarry is also good, and if you're coming up that way and skip Temecula (Red Hawk, Cross Creek, what used to be the SCGA Member's course), then Goose Creek isn't that far out of your way. Also agree with the Arroyo Trabuco recommendation. Haven't lived in the area in years, but Los Serranos in the Chino Hills area has (or had) two good tests. Mountain Meadows in Pomona used to be pretty good as well, but I can't attest to current quality.
All of the public golf caveats with regards to conditions and pace of play apply, especially to Santa Anita and San Clemente, which are both munis (SA is county owned).