News:

Welcome to the Golf Club Atlas Discussion Group!

Each user is approved by the Golf Club Atlas editorial staff. For any new inquiries, please contact us.


TEPaul

Peer Review
« on: June 02, 2010, 09:21:43 PM »
How do the contributors on GOLFCLUBATLAS.com feel that peer review can be maximally effective to any or anyone's opinions or formally framed positions (such as essays, articles and "In My Opinion" pieces) on matters to do with golf course architecture or particularly the history of golf and golf course architects and architecture?

I suppose I'm talking about some process or even protocol that may be somewhat more formal or formalized than just a streaming Discussion Group (DG) like this one, as good and perhaps unique as this one is and has been over the years.

JC Jones

  • Karma: +0/-0
Re: Peer Review
« Reply #1 on: June 02, 2010, 09:24:50 PM »
How do the contributors on GOLFCLUBATLAS.com feel that peer review can be maximally effective to any or anyone's opinions or formally framed positions (such as essays, articles and "In My Opinion" pieces) on matters to do with golf course architecture or particularly the history of golf and golf course architects and architecture?

I suppose I'm talking about some process or even protocol that may be somewhat more formal or formalized than just a streaming Discussion Group (DG) like this one, as good and perhaps unique as this one is and has been over the years.

I find that any "work" beyond a stream of consciousness post is often better with, at a minimum, edits from two people as qualified or more qualified in the particular area than I am.
I get it, you are mad at the world because you are an adult caddie and few people take you seriously.

Excellent spellers usually lack any vision or common sense.

I know plenty of courses that are in the red, and they are killing it.

JESII

  • Karma: +0/-0
Re: Peer Review
« Reply #2 on: June 02, 2010, 09:31:16 PM »
Tom,

Even within the small group of GCA'ers, there's only a small percentage that actually formally frame their positions...that doesn't really answer your question. The framework here (1500 contributors and some larger number of interested observers) gives that micro-group of people that put in the time and energy to produce an In My Opinion, or some other type of essay, an interested, occasionally educated audience of peers.

I think GCA.com is a pretty good source for peer review on this topic.

TEPaul

Re: Peer Review
« Reply #3 on: June 02, 2010, 10:02:20 PM »
"I think GCA.com is a pretty good source for peer review on this topic."


Sully:

By that I assume you mean the GOLFCLUBATLAS.com Discussion Group (DG) is a pretty good source of peer review for any and all "In My Opinion" pieces on this website.
 
 

Peter Pallotta

Re: Peer Review
« Reply #4 on: June 02, 2010, 10:04:17 PM »
Tom - when I was young, I had an image of the peer review process (in matters of science, history, ethics) as a rigourous and stringent and objective sifting of the wheat from the chaff in the disinterested pursuit of truth; and of the peers themselves as wise and good and noble, engaged in the process only in the slim hope that they somehow were contributing to the common welfare. Now, a couple of decades on, and for better or worse (and if you said for worse I wouldn't argue it) I think the whole thing one big racket - a con job, and a nest of vipers. Best (or should I say, least bad, and with John Stuart Mills) is to let the marketplace decide. An essay arguing X is followed (if anyone cares enough) by an essay claiming Y; and a book that says No is countered by another shouting Yes. And so on it goes - the process feeding on and turning back on itself in a dialectic that has at least one thing going for it, ie it doesn't promote itself as anything more (or better and truer) than what it is: the manifestation of countless egos and selves trying to express themselves and to shape their world and life and times.

Peter  
« Last Edit: June 03, 2010, 09:42:44 AM by PPallotta »

JESII

  • Karma: +0/-0
Re: Peer Review
« Reply #5 on: June 02, 2010, 10:10:37 PM »
"I think GCA.com is a pretty good source for peer review on this topic."


Sully:

By that I assume you mean the GOLFCLUBATLAS.com Discussion Group (DG) is a pretty good source of peer review for any and all "In My Opinion" pieces on this website.
 
 

No - my personal opinions in a thread on Manufacturer's are not worthy of peer review (discussion, hopefully...peer review, no!), but someone's My Home Course essay or In My Opinion piece is worthy of consideration.

TEPaul

Re: Peer Review
« Reply #6 on: June 02, 2010, 10:16:12 PM »
"No - my personal opinions in a thread on Manufacturer's are not worthy of peer review (discussion, hopefully...peer review, no!), but someone's My Home Course essay or In My Opinion piece is worthy of consideration."



Why do you say that about a course like Manufacturers?

Is it because you feel you don't know the course well enough to have a really informed opinion of it or is it because you feel you don't know the architectural history of Manufacturers well enough?

Kirk Gill

  • Karma: +0/-0
Re: Peer Review
« Reply #7 on: June 02, 2010, 10:16:50 PM »
Peer review of written works implies that the writer and those performing the review are both qualified in their particular field. I'm sure that there are lots of people who participate on this site that wouldn't necessarily consider themselves qualified to participate in a peer review of essays on the history of golf course architecture.

But, speaking strictly for myself, that doesn't stop me from opening my big fat mouth.
"After all, we're not communists."
                             -Don Barzini

Colin Macqueen

  • Karma: +0/-0
Re: Peer Review
« Reply #8 on: June 02, 2010, 10:23:42 PM »
Jim S,

I agree with you entirely.  Having read many of the My Home Course essays and In My Opinion pieces I actually thought that there had been some review process (though maybe not peer review).  There seems to be a fairly similar style from piece to piece so is there an editorial review by Ran and his cohorts?

The Hielander
"Golf, thou art a gentle sprite, I owe thee much"
The Hielander

Steve Lang

  • Karma: +0/-0
Re: Peer Review
« Reply #9 on: June 02, 2010, 10:40:37 PM »
 8) As a scientist (BS ChemEng & MS Env Eng) i strongly concur with Peter & Jim about formalized peer review on gca.com, I'd simply call it a real lead balloon or buzz killer.  

There are 1500 peer reviewers on gca.com, like it or not, distributions of newbys to over-posters (definitely not gods) and many spectrums of intelligence.  That's the perfect group for this medium and its bandwith.

Though I've faithfully read several peer reviewed science jounals now for decades (facts, figures, tables, analyses, hypothese, inductions, conclusions, deductions) they all evolve as the peer review or editorial panel sees fit.  My favorite journal has an annual "Critical Review" wherein recognized experts all address a general topic from their personal specific perspectives, and that is editted mainly for presentation.

I've always thought that mining the information of gca.com was the best thing going for this professional student..  If only I would properly invest some time, it could actually be scholarly.. but that's no fun.. yet!

having spoken with George Bahto, i will defer to his knowldge and opinon on     CBM!
« Last Edit: June 02, 2010, 10:42:45 PM by Steve Lang »
Inverness (Toledo, OH) cathedral clock inscription: "God measures men by what they are. Not what they in wealth possess.  That vibrant message chimes afar.
The voice of Inverness"

JESII

  • Karma: +0/-0
Re: Peer Review
« Reply #10 on: June 02, 2010, 10:44:58 PM »

Why do you say that about a course like Manufacturers?


Only because there was a thread on it today and I made a couple posts. Those posts are not worthy of any peer review, but if I were to attempt a detailed essay on Manufacturer's that might be.


Colin,

I'm not sure of the process, but I think Ran prefers a reasonable level of writing skill with submitted pieces.


Kirk,

I think the demographic in here qualifies GCA as a reasonable place for peer review...we're not curing cancer after all, and none of us can manage to keep our fat mouths closed...
« Last Edit: June 02, 2010, 10:46:56 PM by Jim Sullivan »

Doug Braunsdorf

  • Karma: +0/-0
Re: Peer Review
« Reply #11 on: June 02, 2010, 11:13:38 PM »

Kirk,

I think the demographic in here qualifies GCA as a reasonable place for peer review...we're not curing cancer after all, and none of us can manage to keep our fat mouths closed...

 Jim, I think that's exactly why, IMO, essays and more formal pieces, but even posts in some cases, may benefit from this peer review or more formalized process.  Or, the articles and all parties involved, both directly and tangentially, may benefit.  What I said isn't a shot on Kirk--I don't know him--but what he said, regarding keeping [his] mouth closed, gets to the whole crux of the situation.
"Never approach a bull from the front, a horse from the rear, or a fool from any direction."

Kirk Gill

  • Karma: +0/-0
Re: Peer Review
« Reply #12 on: June 02, 2010, 11:53:22 PM »
What I was trying to say, other than to get a cheap laugh, was that there's a difference in my mind between 1500 interested parties (including many varied levels of expertise) expressing their opinion vs. a peer review by experts. On the other hand, there are many threads here that I read with a high degree of interest that I'd never consider contributing to because I wouldn't consider myself qualified, and certainly there is enough experience and knowledge amongst the members of this site that a real peer review could be achieved. But besides expertise, a peer review seems to me to imply impartiality and a somewhat dispassionate review. That may be a tougher nut to crack.
"After all, we're not communists."
                             -Don Barzini

Phil_the_Author

Re: Peer Review
« Reply #13 on: June 03, 2010, 12:48:25 AM »
A real-world example that I believe will show that "Peer Review" for In My Opinion articles are more than impractical, they are impossible.

I intend to submit an In My Opinion piece about the evolution of a particular golf course to Ran later in the year. As I am the ONLY person who has been given access to this club's records including their board minutes, who could possibly be considered anything approaching my "peer" on this subject?

There is a big difference between "peer review" and the editorial process and even that isn't actually done with these pieces. There is no "peer review" on this site in any sense of the word despite the comments above. Their is critique, both learned and opinioned, but no peer review, which is as it should be...


Ian Andrew

Re: Peer Review
« Reply #14 on: June 03, 2010, 08:15:37 AM »
A real-world example that I believe will show that "Peer Review" for In My Opinion articles are more than impractical, they are impossible.

I intend to submit an In My Opinion piece about the evolution of a particular golf course to Ran later in the year. As I am the ONLY person who has been given access to this club's records including their board minutes, who could possibly be considered anything approaching my "peer" on this subject?

There is a big difference between "peer review" and the editorial process and even that isn't actually done with these pieces. There is no "peer review" on this site in any sense of the word despite the comments above. Their is critique, both learned and opinioned, but no peer review, which is as it should be...

Not if you share your research.
When it was done in the early days of the "interweb" ;D - all the research accompanied the article.

Every once in a while somebody has found something I have not, if I never asked, I never would have found out.
« Last Edit: June 03, 2010, 08:20:28 AM by Ian Andrew »

Phil_the_Author

Re: Peer Review
« Reply #15 on: June 03, 2010, 12:56:13 PM »
"Not if you share your research. "

And there's the rub Ian. This particular Club that I refered to gave me access with a few restrictions, primary among them is that I am not allowed to reveal direct quotes from Board minutes in anything I may publish.

There are some amazing surprises in what happened there that the Club had no idea about and that golf historians would dearly like to know. So, I can publish the information to let them know OF these events but not give the PROOF.

How can that be "reviewed" by anyone then?

There are times when TRUST in the researcher MUST be granted without review. It is done all the time on here by the way. How often are old newspaper articles posted as proof of a fact or idea? The information presented in the article isn't judged beforehand by those reading it on here but is accepted as fact and proof by its very nature, yet many, many of these articles contain inaccuracies and outright mistakes. For example, I can show you a newspaper article from 1928 which states that Tilly designed Olympia Fields. Not only did Tilly not do that, I have never been able to even find a mention of him having ever even been to the club!

Also, a proper "peer review" panel is made up of recognized experts on the subject that the article is discussing. Once again, how can this be accomplished on here? For example, taking one of our more famous articles that produced more than its share of controversy, David Moriarity's Merion essay.

At the time before it was put up on site, which is when it would have to be "peer" reviewed, who on this site would have been asked to do it? Can you imagine any situation where Ran would not have had Wayne Morrison and Tom Paul be involved because of their background's with Merion and Flynn? Of course he would. What chance would there have been then of it being "accepted" for publication as an In My Opinion piece?

That is NOT criticism of Wayne and Tom, just recognition that they would have rejected it to Ran because they would have claimed that its research was flawed and conclusions were incorrect. Despite the anger and vitriol of the personalities in the discussion of it through those many long months and maybe even years(?), even Tom Paul has stated that information unknown to all came out of those heated discussions (arguments). That would not have happened if there had been a peer review system to the loss of all.

Another example, my piece on the life of A.W. Tillinghast. Who possibly could have done any sort of peer review of that? Frankly speaking, most "golf historians" are still influenced by Frank Haniggan's 1974 USGA article and would have cited that as "proof" that my piece should not have been accepted for publishing, yet I am the only person alive, including Frank Hannigan, to have personally interviewed EVERY living family member who was alive when Tilly was. Doing this brought to light a tremendous amount of information that was missed and so directly contradicts a number of Frank's conclusions. In fairness to him he trusted the one family member he spoke with for his information. That family member trusted his mother, Tilly's daughter, who told him the answers he gave to Frank. Doing so seemed logical at the time but would lead to long-held inaccuracies. That is why Frank's source, Tilly's grandson, after reading my finished manuscript of the biography I wrote before it was published, would send me a letter stating that not only was he unaware of much of what the other relatives stated, but that he believed that there was no one alive who could now claim a greater knowledge of his grandfather than me.

Yet there is considerable question as to whether or not my piece would have survived a "peer review" process for publication on GCA if it had gone through it because those who would have done the review process would have rejected a number of my conclusions as they flew in the face of "known facts."

I'm sorry Ian, but I must disagree with any and all who think it is even possible for a peer review of proposed "In My Opinion" pieces or that it should be done at all. We must put some faith and trust into the process that Ran has set up and in Ran's ability to accept and/or reject these pieces.

Now a nice discussion, even argument about any that are published is proper and fair game for all on the discussion board. The problems have been the personal attacks that have occurred. These are what need to be eliminated.


JESII

  • Karma: +0/-0
Re: Peer Review
« Reply #16 on: June 03, 2010, 01:53:17 PM »
Philip,

I would suggest that you could select a handful of people on here to review your essay and discuss it with you prior to making it public. This may not be scientific Peer Review, but like I said, we're not curing cancer.

I am not suggesting that you do this, just that it is a way for you to experience the process.

The thoughtful, objective feedback you would get in the formal Review process can work, and the folks hanging around GCA.com seem as well qualified as any to help, if desired.

Re: Merion and DM, I don't think Ran would have selected Wayne and Tom because a key component of the Review would seem to be objectivity. While I know they are honest, I don't think they are objective regarding Merion and David's opinions.

Dale Jackson

  • Karma: +0/-0
Re: Peer Review
« Reply #17 on: June 03, 2010, 01:57:37 PM »
How do the contributors on GOLFCLUBATLAS.com feel that peer review can be maximally effective to any or anyone's opinions or formally framed positions (such as essays, articles and "In My Opinion" pieces) on matters to do with golf course architecture or particularly the history of golf and golf course architects and architecture?

I suppose I'm talking about some process or even protocol that may be somewhat more formal or formalized than just a streaming Discussion Group (DG) like this one, as good and perhaps unique as this one is and has been over the years.

I read Tom's initial post with interest and hoped for some interesting comments and have not been disappointed.   A few observations.

Some have taken Tom's initial suggestion to involve a process similar to formal peer review, and I do not believe that is what he had in mind.  (TEP will correct me if I am mistaken, I am sure  :))

A couple of earlier comments mentioned that most on the DG group were not able to comment with any degree of knowledge and I believe that is true.  It is equally true that many are qualified, based on their own research and writings, to offer an experienced and critical eye to help improve the various articles published in the "In My Opinion" section.

Philip Young notes that, in some cases, the writer is uniquely qualified in his area of expertise and it is difficult to have knowledgeable review of such writing.  While I agree to a large extent, I would suggest that all writers can be helped be an "editorial" review of their material.

Speaking personally, I have one article on the IMO section, and it is my intention to prepare one or two more in the next year or so.  My areas of interest and expertise are in the courses and architecture of A.V. Macan.  I would venture that, with the exception of Jeff Mingay, I am the only one on this DG with the interest and knowledge to write on those subjects and having a "peer review" in the traditional meaning of the phrase is not possible.  However, when I am ready to forward my next writings to Ran and Ben for publication, I would welcome a review by some of the others, such as Philip Young, Tom Paul, Ian Andrew, Mark Rowlinson and several others, prior to publication.

I would suggest an informal and voluntary review process could be implemented for those who wish to take advantage of it.  Perhaps those of us who who have experience in writing and publishing could volunteer to be on an "editorial" list, and, as others wish to take advantage of the critical eyes of those on the list, they could forward their material to whomever they think could help with the quality of their writing.
I've seen an architecture, something new, that has been in my mind for years and I am glad to see a man with A.V. Macan's ability to bring it out. - Gene Sarazen

Phil_the_Author

Re: Peer Review
« Reply #18 on: June 03, 2010, 02:42:31 PM »
Dale and Jim,

From having spoken about this issue with Tom in the past, the impression he gave me is that he favors a more formalized review process. I am certainly in favor of anyone who would like to have an In My Opinion piece send an advance copies to thers for comments, suggestions, etc... I did that with my Tilly piece and David Moriarity showed his essays to some before it was published. In fact there was quite a bit of discussion on site about the coming essay and the information it might contain because he had.

Dale, I would be more than happy to take a look at anything you write for publication on here or elsewhere...

JESII

  • Karma: +0/-0
Re: Peer Review
« Reply #19 on: June 03, 2010, 03:28:01 PM »
Phil,

I wouldn't argue for one or the other...and I think Tom would clearly prefer a more formal process...but I wonder why. He hasn't written anything for the site that I am aware of.

I don't want Tom's motivations to guide this discussion so I'll just say that David did it the wrong way and so did Tom and Wayne.

How would you address your own concerns about having no peers when the topic is Tillinghast?

Phil_the_Author

Re: Peer Review
« Reply #20 on: June 03, 2010, 05:01:45 PM »
Jim,

I wouldn't say that I have no peers, I think that there are a very few, maybe a handfull only, who have close to the information on Tilly that I have, and these aren't anywhere near as organized as my collection is.

I have another advantage as I am responsible for answering questions sent to the Tillinghast Association that members, clubs and individuals have. That has enabled me to build up many close relationships to clubs, with members and even architects who are seeking information. As a result I protect these relationships and approach them all with a sort of Doctor/Patient approach, revealing only those things publicly that they specifically allow me to from what they've shared.

That said, I don't know everything about Tilly and often a question is sent my way or other information sent to me from a friend "for my collection" that leads to new discoveries. For example, earlier today Joe Bausch emailed me about some Board minutes that he located for a Philadelphia area club that "Listed as a person being paid 25 bucks in late 1912, the time the club was looking for land for a new course, is a "Mr. Tillinghaust"."

Now if this is Tilly, and it appears to be so (Sorry Joe I'll answer your email in more detail later), and he gave advice that led to his designing a course for them, then he will have discovered a heretofor unknow Tilly. That will actually make 4 NEW Tilly's discovered in the last two weeks because of questions like his. We are working hard at verifying each of them and are already 99% certain of the other three. All will be announced in an upcoming issue of Tillinghast Illustrated.

Now all that doesn't really answer your question. Let me put it this way; I am constantly approaching questions and anything I write in the same manner, that is, that I don't know the answer before writing. I try to continually re-research when questions are asked. Case in point, North Shore.

When the Tillinghast accreditation was again brought up I got involved in the discussion. The brief club history done by the member a number of years ago which stated that Tilly had designed the course had served as the proof for the club for a number of years now. As the discussion progressed and different sides expressed their strong opinions (including me) no progress was being made. It appears now that I was the only person aware that the original board minutes were at the New York Historical Society. To end what was an ever-growing acrimonious debate I mentioned this to Steve and asked if he'd like to go with me to see them during a planned New York trip several months later. Fortunately he decided not to wait and went in himself and made the surprising discovery that it is a Raynor design and that Tilly wasn't involved.

I very easily could have simply fallen back on my "previous knowledge" and taken the Tilly designed it stand. Anyone seeking better information would have been forced to go to the club itself and, if they were granted access, would have run into a brick wall as the minutes aren't there and no one there was aware of the archive at the New York Historical Society.

I am not claiming to have discovered the answer; Steve did and ALL the credit for doing so belongs to him and his impatience and refusing to wait for me to come up from Atlanta. That is the nature of this research and discovery process. It is also why there are times where what has been written in the past needs questioning and looking at while there are some things written today that demand acceptance as established fact because of what was written and by whom.

So, again, and I'm sure I may have confused you and everyone else as this is a very difficult question to answer. No, I am not the be-all, end-all, know-all on Tillinghast. But there are very few whose opinions match the knowledge, personal archives and resources that I have and have access to...

Sorry for the lousy answer...
 


Steve Lang

  • Karma: +0/-0
Re: Peer Review
« Reply #21 on: June 03, 2010, 05:04:28 PM »
 8) Essays could certainly be reviewed, but who's to be executive editor???

No way would "In My Opinion" pieces be able to be true to their authors if others reviewed and proposed to change things.. that's one step short of thought control and/or censorship
Inverness (Toledo, OH) cathedral clock inscription: "God measures men by what they are. Not what they in wealth possess.  That vibrant message chimes afar.
The voice of Inverness"

JESII

  • Karma: +0/-0
Re: Peer Review
« Reply #22 on: June 03, 2010, 05:09:51 PM »

Sorry for the lousy answer...



That was a really lousy answer...but also a perfect one.

My real question is, if you're looking for some form of review prior to making your Tilly essay public (not that I think this should be required), what better place to find qualified people than some of those posting here on GCA? None of them will know as much about Tilly as you do/will, but the subject matter will not be foreign to them.

Maybe I just don't understand the scientific world/usage of Peer Review.

Peter Pallotta

Re: Peer Review
« Reply #23 on: June 03, 2010, 05:32:57 PM »
But my question again (and this time phrased as a question), off Jim's last post:

To what end?

What do we think this peer review process (here, with posters qualified or not, reviewing an essay) will or can lead to?

To a better essay? Perhaps.Or not.

To a more authoritative essay?  Well, what does that mean, authoritative? In whose eyes? And for what purpose - so that it can somehow be deemed authoritative and worthy of publication? (Who'd publish it?)

And, if we could somehow imagine a genuinely authoritative text/essay, would it mean that any subsequent and/or contrary essay is automatically deemed 'un-authorized', to be spat-upon and thrown into the dust-heap?

How have we furthered anything at all?

Or am I misunderstanding all of this - and what we actually want is a (slightly tense, sometimes hurtful) work-shop, where we toss around ideas and a writer takes some and ignores others and we throw in our two cents just because, well, just because we're a discussion group and that's what discussion groups do?

If it's that, fine. But then let's not even mention the term "peer review" -- because if I've misunderstood it's understandable: peer review is a pretty specific term.

No.

My view is that a place like the USGA Archive can gather over time all sorts of documents and photos and texts and essays and a library fuill of old newspaper accounts (all of them un-authorized), and interested parties can read one thing and another and make up their minds.

Over time, a 'conventional wisdom' will develop.

And then, years later, someone will come along and challenge that convention.  

Peter
« Last Edit: June 03, 2010, 05:35:16 PM by PPallotta »

Dale Jackson

  • Karma: +0/-0
Re: Peer Review
« Reply #24 on: June 03, 2010, 06:20:09 PM »

To what end?

What do we think this peer review process (here, with posters qualified or not, reviewing an essay) will or can lead to?

To a better essay? ...

To a more authoritative essay?  ...

Or am I misunderstanding all of this - and what we actually want is a (slightly tense, sometimes hurtful) work-shop, where we toss around ideas and a writer takes some and ignores others and we throw in our two cents just because, well, just because we're a discussion group and that's what discussion groups do?


Peter,

In my view the purpose is to improve the quality of what is published on this site.

Can any writing be improved by independent eyes?  Of course.

And does the level of improvement increase when eyes knowledgeable in the subject matter are involved?  Again, I suggest, of course.

As for the Discussion Group, that is a separate feature of this web site.  When I posted my article, it was not to offer it up to 1,500 people on the DG for comment, but to publish it on a web site that is seen by a broader audience as being a resource for a wide range of information on golf architecture.


... But then let's not even mention the term "peer review" -- because if I've misunderstood it's understandable: peer review is a pretty specific term.


I agree, this is not technically "peer review", but frankly I do not care what it is called, I would just like to have the resources available to me to review my writing.
I've seen an architecture, something new, that has been in my mind for years and I am glad to see a man with A.V. Macan's ability to bring it out. - Gene Sarazen