If you left Forest Dunes anytime in the early to mid evening on Friday (say after 36 holes, so 6 or 7 pm), you could make the four hour drive to Greywalls (it's no more than that from FD), play there in the morning, and then make the big drive to Arcadia (about six and a half hours).
But Arcadia afternoon rounds are notoriously slow...what I'd do in that case is drop one of the rounds at FD for an earlier Arcadia round. Maybe Arcadia could be an afternoon round on Friday, and then follow it up with a late drive to Greywalls that night, or a there/play/and back drive on Saturday, taking all the time you need to get back downstate on Saturday night.
The trip to Greywalls is definitely worth it if you can swing it. There's always a way.