Ward,
For me, the most difficult par 4's aren't always the hardest par 4's.
For me, the most difficult par 4's are challenging par 4's that happen to be the opening hole, your introduction to the golf course.
I'm addressing your question in the context of a round that has meaning, a qualifying round, or a medal play round in a tournament.
Often, in addition to the difficulty of the hole, there's the pressure of getting off to a good start as opposed to a bad one, which often sets the tempo for the entire round. There's the uncertainty of taking your warm up on the range out onto the course where it now counts.
Holes like # 1 at WFW and Wannamoissett would be good examples.
Once you're into the round the first tee jitters or pressures diminish.
The other most difficult par 4, for me, is a challenging par 4 at # 18 where you have a good/great round going, or are in a match where # 18 will decide the match. # 18 at WFW, Riviera, ANGC, Seminole, PV, Merion and Shinnecock might be examples of those holes.
When you've won or lost your match 3 & 2, those holes are a lot easier than when you're all square on the 18th tee.