From a naive perspective, on a relatively flat green, I think it's safe to say that faster greens are "more difficult" simply because the amount of error in a shot to keep it within three feet is smaller.
This isn't the end of the story, however, because you also need to account for contouring. If the contouring is not matched well with the faster speed, then the pinnable areas get so small that putting down a contour become simply getting to ball to effectively stop at the top of the contour and have it run down to the pin for an easy tap in. Whether or not this is "more difficult" is debatable. On the one hand, it's simply like putting a shorter distance, and allowing the ball to run out, but on the other hand, it still requires the player to read the run out.
I guess the thing I would say is most important, is that (it seems to me at least) fast and slow greens putt very differently. It feels like the skill required to curl a putt to the hole on a slower green is just a fundamentally different putt (and line) than the same putt on a fast green, where the ball will trickle down from above the hole on it's own. I don't think one or the other is significantly more difficult (though I tend to struggle more on faster greens), but the feel is wildly different to me.