Anirudh Krishna (Duke University) at Ideas for India: The root of poverty: Ruinous healthcare costs.
James Choi at : How Noisy is Economics/Finance Peer Review?, with excerpts from this paper. I was quite surprised to read this:
For economics journals, when two referees are consulted, the top-10p [percentile] paper receives two rejects with probability 14%, one reject and one non-reject with probability 47%, and two non-rejects with probability 40%. With three referees, the top-10p papers receives a majority of reject recommendations with 30% probability, a majority of non-reject recommendations with 70% probability.
Annie Murphy Paul in NYTimes: It’s Not Me, It’s You, an essay on intelligence and stereotype threat.
Felix Salmon: What education reformers did with student surveys.
Sunday, November 25, 2012
Links
Filed under: Economics, Education, Health, Psychology, Publish/perish, Sociology
Subscribe to:
Post Comments (Atom)
1 Comments:
Consider two different methods whereby authors select the journal to which their paper should be sent:
(i) the names of all journals are put in a hat, a name is drawn at random and the paper sent to that journal
(ii) the authors make an implicit judgement about the "quality" of their paper, the time taken for processing by a journal and many other factors and choose the journal accordingly.
Note that you would see far more "agreements" in (i) as opposed to (ii). In (i), you would have a lot of cases where "bad" papers are sent to "good" journals and vice versa. Those are precisely the cases where you'd expect to see agreement amongst referees. However, (ii) is how authors actually operate. The raw data thus overstates the level of "disagreement" among economics referees.
Of course, what adds to the mix is the nature of the subject itself. In general, the "quality" of an Economics paper is not fully apparent, even to the author. Given this, the disagreement amongst referees is not surprising. It is worth noting that a lot of times, the disagreement is not about the validity of the paper's results as such but whether the paper deserves to be published in the concerned journal or should go to a lesser journal.
What would be interesting to know is how Economics compares to other subjects in this regard.
Post a Comment