Ryan,
I've played both, Victoria and Colwood, over the past month and a bit. They're both fine golf courses, but completely different experiences; which is really cool.
Ideally, you should play both
The posts above sum things up pretty well. Basically, if you're into (some) quirk and tradition, you should see Victoria; if you're a more conventional thinker, you'll probably prefer Colwood (longer and a bit more straightforward). Settings are completely different as well. There are some neat, old trees at Colwood, but nothing compares to playing golf by the sea at Victoria on a beautiful day.
Hope that helps.
P.S. I'm heading to Victoria next week. We're scheduled to do quite a bit of restorative-based work on the course over the next few months and more. Be sure to say hi if you end up at VGC.