Wednesday, August 25, 2010

Interesting take on peer review & openness from outside the sciences in @nytimes

I assume many supporters of open science may have seen this already but if not it is worth a look.  The New York Times had an interesting article on Monday by Patricia Cohen: For Scholars, Web Changes Sacred Rite of Peer Review.

The article starts off with a familiar refrain
For professors, publishing in elite journals is an unavoidable part of university life. The grueling process of subjecting work to the up-or-down judgment of credentialed scholarly peers has been a cornerstone of academic culture since at least the mid-20th century.
It follows with a very important discussion focusing on how the web can transform scholarly publishing.  For example:
... scholars have begun to challenge the monopoly that peer review has on admission to career-making journals and, as a consequence, to the charmed circle of tenured academe. They argue that in an era of digital media there is a better way to assess the quality of work. Instead of relying on a few experts selected by leading publications, they advocate using the Internet to expose scholarly thinking to the swift collective judgment of a much broader interested audience.
This likely will sound very familiar to those who have read my blog, those who follow the discussions on peer review, or those with a pulse in the scientific community.  But there is a catch that caught me off guard here and might surprise many of you.  This catch is highlighted by the fact that the article was in the Arts section of the Times.  You see, the article was about transformation in the humanities.  Seems as though there is an almost completely parallel universe there where peer review and publishing and sharing are all getting re-evaluated.

This is yet another case of why we need more cross talk between the arts/humanities and the sciences.  For example, the article discusses how the journal Shakespeare Quarterly is becoming the first humanities journal to "open its reviewing to the World Wide Web." They even recently conducted an experiment in fully open review where four preprints were posted on the web and feedback was solicited.  The feedback was then used by editors to guide the revision of the preprints to become published articles.  Sounds a lot like Biology Direct.  Note however, that they are not talking about publishing the final articles in an open access manner (see discussion of this on BigThink here) - but more about engaging the broader audience in commentary before an article is published.

The article does suggest that perhaps the humanities are lagging a bit behind the sciences in experimenting with new forms of peer review but I think that is OK.  We desperately need new experiments and ideas in this arena.  Peer review, at least the way it operates right now, has many problems.  I think there must be many better ways to go about things.  And thus cross pollination across fields from arts and humanities to economics to physics to life sciences is a good thing.

Interestingly Cohen identifies what she considers to be the most daunting obstacle to opening up review:
peer-review publishing is the path to a job and tenure, and no would-be professor wants to be the academic canary in the coal mine.
I think this is the same main obstacle in the sciences.  That is, it is our system of promotion and tenure and hiring that is the main roadblock.

Finally, I note that though the article was focusing mostly on opening up peer review, it does have some interesting bits on openness in general. In particular, there is a great line at the end from Dan Cohen from George Mason
“There is an ethical imperative to share information,” said Mr. Cohen, who regularly posts his work online, where he said thousands read it. Engaging people in different disciplines and from outside academia has made his scholarship better, he said.
I could not agree more.  Seems like the arts and humanities and sciences actually have much much more in common that many might think.

For some related posts from the web see

Some recent web stuff on peer review in the sciences

1 comment:

  1. TILL STANDARDS, SUSTAINABILITY AND SCALABILITY ARE TESTED:
    PEER COMMENTARY IS SUPPLEMENT, NOT SUBSTITUTE, FOR PEER REVIEW

    See: Peer Review Reform: bit.ly/peer-review-reform

    Harnad, S. (1978) Inaugural Editorial. Behavioral and Brain Sciences 1(1).
    www.ecs.soton.ac.uk/~harnad/Temp/Kata/bbs.editorial.html

    Harnad, S. (ed.) (1982) Peer commentary on peer review: A case study in scientific quality control, New York: Cambridge University Press.

    Harnad, Stevan (1985) Rational disagreement in peer review. Science, Technology and Human Values, 10 p.55-62. cogprints.org/2128/

    Harnad, S. (1990) Scholarly Skywriting and the Prepublication Continuum of Scientific Inquiry Psychological Science 1: 342 - 343 (reprinted in Current Contents 45: 9-13, November 11 1991). cogprints.org/1581/

    Harnad, S. (1996) Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals. In: Peek, R. & Newby, G. (Eds.) Scholarly Publishing: The Electronic Frontier. Cambridge MA: MIT Press. Pp 103-118. cogprints.org/1692/

    Harnad, S. (1997) Learned Inquiry and the Net: The Role of Peer Review, Peer Commentary and Copyright. Learned Publishing 11(4) 283-292. Short version appeared in 1997 in Antiquity 71: 1042-1048. Excerpts also appeared in the University of Toronto Bulletin: 51(6) P. 12. cogprints.org/1694/

    Harnad, S. (1998/2000/2004) The invisible hand of peer review. Nature [online] (5 Nov. 1998), Exploit Interactive 5 (2000): and in Shatz, B. (2004) (ed.) Peer Review: A Critical Inquiry. Rowland & Littlefield. Pp. 235-242. cogprints.org/1646/

    Harnad S. (2002) BBS Valedictory Editorial (2002) Behavioral and Brain Sciences 24 users.ecs.soton.ac.uk/harnad/Temp/bbs.valedict.html

    Harnad, S. (2003) PostGutenberg Peer Review the invariant essentials and the newfound efficiencies users.ecs.soton.ac.uk/harnad/Temp/peerev.pdf

    Harnad, S. (2009) The PostGutenberg Open Access Journal. In: Cope, B. & Phillips, A (Eds.) The Future of the Academic Journal. Chandos. eprints.ecs.soton.ac.uk/15617/

    ReplyDelete

Most recent post

My Ode to Yolo Bypass

Gave my 1st ever talk about Yolo Bypass and my 1st ever talk about Nature Photography. Here it is ...