Monday, June 12, 2006

Open Peer Review of Scientific Articles

Over the past 4 years I have provided some online facilitation support to a variety of global scientific collaboration efforts. It was an amazing learning opportunity to see the clash between some of the artifacts of the "culture of science" and the need for boundary crossing collaboration to serve urgent human and planetary (a.k.a. mother nature) needs.

That's why I'm excited every time I seen an innovation that seeks to preserve the best of the scientific culture and move past some of the barriers. The respected international weekly journal of science, Nature, is doing just that with their Peer Review Trial:
"In Nature's peer review trial, lasting for three months, authors can choose to have their submissions posted on a preprint server for open comments, in parallel with the conventional peer review process. Anyone in the field may then post comments, provided they are prepared to identify themselves. Once the usual confidential peer review process is complete, the public 'open peer review' process will be closed. Nature will report on the results after the trial period is over."
One of the barriers I've noticed to knowledge sharing is "publish or perish." The practice of very carefully sharing (or not at all) early data prior to publication has some unintended consequences. It slows down collaboration and potentially, stifles innovation. It creates a competitive scientific market where sometimes we need a collaborative one. The journal peer review process is intended to create rigor and critical thinking so we aren't all shammed by a fakester. But it also create firewalls between information and the public.

So how do we benefit from rigor, collaboration and cooperation?

Nature's experiment with open public peer review looks to test what happens when a private process is made public. This is not the same as ditching the peer review process, but creating transparency and a paralell track of public review.

I'm looking forward to what is learned in this trial for Nature. I'm also looking forward for what it stimulates in the scientific community about the possibilities of openness. To move away from the publish or perish model, or to reimagine it to support cross boundary collaboration where the public good can be served, is a huge change. It would rattle down to the tiny bones of every organization. It would change donor and funding streams. It would change higher education.

1 Comments:

Anonymous TheCRE.com said...

Peer Review Failure at JAMA

The Center for Regulatory Effectiveness www.TheCRE.com has revised the Wikipedia entry on peer review to discuss and document peer review failure at JAMA. The revised entry exposes a politically biased Journal of the American Medical Association (JAMA) article [JAMA, May 24, 2006; 295(20): 2407 - 2410] on the Data Quality Act and atrazine that contains numerous factual errors and misrepresentations. A crucial, obvious error in the article was the assertion that atrazine is being phased out by the European Union because “Atrazine...has been repeatedly demonstrated to be a potent endocrine disruptor....” JAMA’s peer review process accepted this claim even though the Official Journal of the European Union explicitly stated “In the 70s, a political decision was taken to reduce to ‘zero’ the presence of pesticides, independent of their toxicity.” [Emphasis added]

The CRE Wikipedia revisions also include a discussion of the peer requirements imposed on federal regulatory agencies by the Office of Management and Budget. Federal agencies cannot use or rely on scientific information that does not meet the OMB peer review requirements. Many peer reviewed journals do not meet the OMB peer review requirements.

For more information about the failed peer review at JAMA, please contact William G. Kelly, Jr.. For more information about the US government’s peer review requirements, please contact Scott Slaughter.

Wikipedia article on peer review

3:45 PM  

Post a Comment

Links to this post:

Create a Link

<< Home


Full Circle Associates
4616 25th Avenue NE, PMB #126 - Seattle, WA 98105
(206) 517-4754 -