A Case Study in Sociology of Science

In yesterday's post, I linked to a 1992 article in Nature on the months-long ordeal of two chemists who tried to get the Journal of the American Chemical Society to publish their critical commentary on two sloppy papers from Ron Breslow's group.

That article is behind a paywall, unfortunately.

Now, you can still get a sense of that horror story; over at The Curious Wavefunction, Ashutosh Jogalekar has an excellent post on the 1992 Nature paper: The anatomy of peer review: Why airing dirty laundry in public is important.

Ashuosh does a great job of drawing out the key lessons from this grim episode: the role of power in the peer review process, and how internet has made a difference in helping scientists to correct the record quickly through blog commentaries. Go read all of it!

The most remarkable fact about this account is that Nature published it, and in writing it Menger performed a unique and valuable public service. Personally I have never seen such a detailed dissection of peer review described in a major journal. Some people would deplore this public airing of dirty laundry. They would say that none of this can undo what happened, and the only effect of such articles is bad blood and destroyed reputations. I happen to disagree. I think journals should occasionally publish such analyses, because it alerts us to the very human aspect of science. It demonstrates to the public what science is truly like, how scientists can make mistakes, and how they can react when they are corrected or challenged. It sheds important light on the limitations of the peer review process, but also reaffirms faith in its ultimately self-correcting nature. Some people might think that this is a great example of how peer review should not be, but I would like to think that this is in fact exactly how the process works in the vast majority of cases; imperfect, ambiguous, influenced by human factors like reputations, biases and beliefs. If we want to understand science, we need to acknowledge its true workings instead of trying to fit it into our idealized worldview of perfect peer review.

In this day and age, blogs are performing the exact same function as Nature did in 1992, and this is clearly apparent from the latest Breslow brouhaha. Menger and Haim in 2012 would not have to test their patience by trying to publish in JACS for 11 months; instead they could upload their correction on a website and let the wonder of instant online dissemination work its magic. Blogs may not yet be as respectable as JACS, but the recent incident shows that they can be perfectly respectable outlets of criticism as long as the criticism is fair and rigorous. The growing ascendancy of blogs and their capacity to inflict instant harm on sloppy or unscrupulous science should hopefully result in much better self-policing, leading authors to be more careful about what they publish in "more respectable" venues. Thus, quite paradoxically, blogs could lead to the publication of better science in the very official sources which have largely neglected them until now. This would be a delightful irony.