Saturday, May 23, 2015

Bad Incentives are behind Big Science Frauds

Updated with links.

* * *

Adam Marcus and Ivan Oransky, the folks behind Retraction Watch have penned an op-ed in NYTimes: What’s Behind Big Science Frauds? They flag a circle of behaviours to support their thesis that "the incentives to publish today are corrupting the scientific literature"

  • Science fetishizes the published paper as the ultimate marker of individual productivity. And it doubles down on that bias with a concept called “impact factor” — how likely the studies in a given journal are to be referenced by subsequent articles.

  • Journals with higher impact factors retract papers more often than those with lower impact factors.

  • Scientists view high-profile journals as the pinnacle of success — and they’ll cut corners, or worse, for a shot at glory.

  • [The reviewers at top journals] seem to keep missing critical flaws that readers pick up days or even hours after publication.

  • [P]erhaps journals rush peer reviewers so that authors will want to publish their supposedly groundbreaking work with them.

The news that appears to have triggered this op-ed is a recent retraction of a high profile paper. Since the original paper was on a topic of wide interest, it got picked up by many news outlets. Now that many problems with the data presented in the paper have been uncovered, the senior author (Prof. Donald Green, at Columbia) has sought a retraction, the journal (Science) has responded with an expression of concern, and the junior author (Michael LaCour, a grad student at UCLA) says he stands by his study and promises a comphrehensive response by the 29th of May.

Read the whole thing at Retraction Watch: Author retracts study of changing minds on same-sex marriage after colleague admits data were faked.

Here's a round-up of how various news outlets which covered the original paper have responded to the retraction.

* * *

Update (26 May 2015): An Interview With Donald Green, the Co-Author of the Faked Gay-Marriage Study by Jesse Singal in New York Magazine.

How the Gay Rights Canvassing Study Fell Apart by Naomi Shavin in The New Republic.