Stephen Downes
Stephen's Web
Privileged Peer Review - Whose Opinion Counts?
George Siemens,
Connectivism Blog, July 26, 2007.
George Siemens responds to concerns that his work, including his book on connectivism, has not been peer reviewed. My own response is similar to his: peer review is not the only way of ensuring quality, and it's not even a good way of ensuring quality. It takes too long and the process is conducted in secret. And there's no reason to believe that a small group of experts can assess quality any better - and more fairly - than an open and public assessment by anyone who cares to read the item in question.
Comments
Re: Privileged Peer Review - Whose Opinion Counts?
It seems to me you have not thought through well enough the social dynamics of peer review. Editors of journals are not blind. Their reputation is very much at stake in terms of what is published in their journals. Editors almost always "referee" the paper themselves in addition to the blind referees and editors exercise substantial control in the evaluation process via the selection of referees. Further, refereeing itself is an important professional function and most scholars I know take that function quite seriously. And within a field of inquiry the journals themselves are ranked. So peer review or not shouldn't be considered an on/off thing but rather a matter of degree.
Since in discussing learning it is fashionable to focus on outcomes rather than on inputs (or process) why not do the same with quality of published work? Apart from the lags, I agree those are regrettable, do you have other evidence that peer review does a poor job?
It is definitely not perfect and to the extent that authors are taking a fundamentally new approach, there is a greater chance of rejection because the referees and editors don't yet have a good way to evaluate and understand the implications of the work. George Akerlof, Nobel Prize Winner in Economics, had his famous paper "The Market for Lemons" rejected at two or three places before it got accepted in the Quarterly Review of Economics. It was a seminal paper with great influence on the thinking in the profession, but it took a while for those ideas to get absorbed by others in the profession.
This is not to say that items directly published, such as your blog, can't be of quality and the commenting process is certainly an alternative form of validation (or refutation). To me each has usefulness in its own domain and I don't see why it is necessary to conclude that one trumps the other. [Comment]
[Permalink]
[Previous][Next]
Re: Privileged Peer Review - Whose Opinion Counts?
> It seems to me you have not thought through well enough the social dynamics of peer review.
You may disagree with me, but I don't know why you would say I haven't thought it through. I write many papers, I've published both in peer reviewed journals and elsewhere, and I've served on peer review committees. I have also studied social networks, philosophy of science, and more. In other words, I have thought it through - and have drawn the conclusions I have. I could say that you didn't think it through - after all, you disagree with me! [Comment]
[Permalink]
[Previous][Next]
Re: Privileged Peer Review - Whose Opinion Counts?
I did not intend any disrespect. I had hoped that you'd comment on the role of the editor in peer review. [Comment]
[Permalink]
[Previous][Next]
Re: Privileged Peer Review - Whose Opinion Counts?
I think editing is a very separate function from peer review. I have had the fortune to work with some very good editors over the years, and some of my best articles have resulted. But in two of three cases, the editors were not associated with peer review. Moreover, most of my essays that are published in peer reviewed publications manage to do so without any significant contribution from an editor. So it's clear that peer review does not entail editing.
Peer review has to do with the selection process. It is this process I have in mind when I criticize it. Editing has to do with the improvement process. I see no reason why this process cannot stand alone. [Comment]
[Permalink]
[Previous][Next]
Re: Privileged Peer Review - Whose Opinion Counts?
Thank you. I believe there to be a substantial contribution of the editor in (1) selection of the referees and (2) making a decision about the disposition of a paper when the referees disagree. Both of those actions are largely invisible to the author. I had some experience like you, with the editor as collaborator, but only in learning technology pieces, not for economics articles. I agree that is rewarding. I would add that referees can act a collaborator, viewing part of their role to improve the paper, but whether they do so or not depends a lot on how the editor frames the request for the referee report. Of course it also depends on the referee, whether he views the job as an important obligation to the profession or merely as something to get done with.
I made different but related comments directly on George Siemens blog. To me, this is an extremely important issue. I believe that faculty are so tied up in doing their own research and embracing the peer review process both as author and referee, and occasionally as editor too, that by openly arguing against peer review we deliberately put ourselves outside their world. I, for one, am trying hard to get them to enter my world; it is a struggle. And since I see the value both of formal writing and informal writing, I'm quite comfortable arguing that both can coexist. [Comment]
[Permalink]
[Previous][Next]
Your comments always remain your property, but in posting them here
you agree to license under the same terms as this site
(Creative Commons). If your comment is offensive it will
be deleted.
Automated Spam-checking is in effect. If you are a registered
user you may submit links and other HTML. Anonymous users cannot
post links and will have their content screened - certain words are prohibited
and your comment will be analyzed to make sure it makes sense.
