Thursday, March 16, 2006

Broken: Science publishing


Everyone, it seems, has a problem with peer review at top-tier journals. The recent discrediting of stem cell work by Woo-Suk Hwang at Seoul National University sparked media debates about the system's failure to detect fraud. Authors, meanwhile, are lodging a range of complaints: Reviewers sabotage papers that compete with their own, strong papers are sent to sister journals to boost their profiles, and editors at commercial journals are too young and invariably make mistakes about which papers to reject or accept (see Truth or Myth?). Still, even senior scientists are reluctant to give specific examples of being shortchanged by peer review, worrying that the move could jeopardize their future publications.

So, do those complaints stem from valid concerns, or from the minds of disgruntled scientists who know they need to publish in Science or Nature to advance in their careers? "The rising [rejections] means an increase in angry authors," says Drummond Rennie, deputy editor at Journal of the American Medical Association (JAMA). The timing is right to take a good hard look at peer review, which, says Rennie, is "expensive, difficult, and blamed for everything."

What's wrong with the current system? What could make it better? Does it even work at all?

From this article by Allison McCook in The Scientist.

Over at the Seed Magazine, Dave Munger explores another facet of science publishing through this question: " What happens when research bypasses the validation process and goes straight to the public?"

Publishing a study is often a downright lethargic process: Researchers with a hot new result submit an article to a journal, wait a minimum of three months for experts in the field to review the findings and then spend another few months revising to the reviewers' recommendations. When a paper is finally accepted, it is edited again and checked by the authors, only to linger—often for several more months—for its turn at the printer.

The process can take so long that some scientists have decided to avoid it altogether.

Coming to Indian journals, this report has some gory details which are shocking, to say the least. India, as a country, seems to be home to a lot of publications -- that call themselves science journals -- which don't believe in the process of peer review (however flawed it may be, it's still the only one we've got!).

Among the analysed 1835 journals, 145 (7.9%) have listed their editorial board; however, we realize that many journals never use their expertise for review or content optimization. ... [Just] 154 journals (8.39%) come under the category ‘peer reviewed’. About 331 journals are available online.

The authors of this study, P. Pichappan and G. Buchandiran (Department of Library and Information Science, Annamalai University, Chidambaram, Tamil Nadu), conclude their short article with these observations:

We are concerned with poor peer-reviewing practice of Indian journals. Indian libraries should subscribe to the reviewed journals only, so that the other journals realize the potential and value of the review system. Unless the institutions insist that their scientists and faculty members opt for publications in peer-reviewed journals and consider the publications in peer-reviewed journals only, Indian journals will continue to live in a constricted circle.

1 Comments:

  1. Anonymous said...

    Peer Review Failure at JAMA

    The Center for Regulatory Effectiveness www.TheCRE.com has revised the Wikipedia entry on peer review to discuss and document peer review failure at JAMA. The revised entry exposes a politically biased Journal of the American Medical Association (JAMA) article [JAMA, May 24, 2006; 295(20): 2407 - 2410] on the Data Quality Act and atrazine that contains numerous factual errors and misrepresentations. A crucial, obvious error in the article was the assertion that atrazine is being phased out by the European Union because “Atrazine...has been repeatedly demonstrated to be a potent endocrine disruptor....” JAMA’s peer review process accepted this claim even though the Official Journal of the European Union explicitly stated “In the 70s, a political decision was taken to reduce to ‘zero’ the presence of pesticides, independent of their toxicity.” [Emphasis added]

    The CRE Wikipedia revisions also include a discussion of the peer requirements imposed on federal regulatory agencies by the Office of Management and Budget. Federal agencies cannot use or rely on scientific information that does not meet the OMB peer review requirements. Many peer reviewed journals do not meet the OMB peer review requirements.

    For more information about the failed peer review at JAMA, please contact William G. Kelly, Jr. wgkelly@tetontel.com. For more information about the US government’s peer review requirements, please contact Scott Slaughter, Slaughter@mbsdc.com.

    Wikipedia article on peer review