There are literally 1,000 courses in Japan with two greens on EVERY hole, and most of them stink.
On rare occasion, there might be a place where it makes sense to build two separate greens -- if you love both green sites, if they truly add variety of shots played, and if they both fit into the flow of the next hole.
The architect who I have seen use them most often in recent years is Tom Fazio ... he picked up the idea when he built an alternate green for the 8th at Pine Valley, and decided he could use the idea to make a very very small green for a short par-4 (or a par-3) with an alternate green so he wouldn't have to worry about wear and tear on the small one. Shadow Creek, Pelican Hill, and Pronghorn all have versions of this hole, and I'm sure he's done it elsewhere as well.