Two links:
Arunn has a post on Quantifying Research Quality through Article Level Metrics.
The beginning of the end for impact factors and journals, a neat online article by Richard Smith [3], explains the newly introduced ALM indices with examples. Another recent article published in PLoS by Cameron Neylon and Shirley Wu [4] discusses the pros and cons of the newly introduced ALM indices. But both these articles leave out in their discussion, certain key journal requirements for proper functioning of the proposed ALM and their related shortcomings.
Also, journal impact factor is being seen as a very poor measure of article impact. One distinction is essential in such generalization. Because we are able to debunk the efficacy of impact factors, we are not debasing the reputation earned by research journals.
In this article, we discuss the efficacies of the proposed ALM indices, journal impact factor contrasted with the prevailing journal reputation and related issues in detail. In the summary, we provide possible rectification measures for ALM.
- Sachin Shanbag has one on Quantitative v/s Qualitative Evaluations: Impact Factors and Wine Experts:
I think they are a lazy substitute for actually reading a person's research and evaluating its worth individually. While it is fashionable, and getting increasingly so, I've never really been a big fan of using purely quantitative factors to measure the worth of an individual, university, or country.
You wouldn't necessarily think that the musician who sells the most records, or has the most covers made is necessarily the best (that would rate the likes of Back Street Boys over bands like Dream Theater).
2 Comments:
Hello Abi,
I stumbled onto your blog while researching the antics of the great, the one and only V. Shiva Ayyadurai, and I can only say, if THIS is the best thing that VSA ever does for me, it will still be far more than enough to justify his miserable existence.
May I draw your attention to something I wrote for Current Science many years ago? The URL is here:
www.ias.ernet.in/currsci/jun10/articles2.htm
What I was trying to say is this: (i) All decisions on promotion, tenure, awards etc. are by nature subjective. Any attempts to "quantify" such decision-making will only serve to mask the subjectivity of the process. (We simply transfer the subjectivity from the *application* of the criteria to the *enunciation* of the criteria.) (ii) The fact that the decisions are inherently subjective IS NOT A REASON to doubt the integrity of the people making the decisions. Most of the time they are doing the best they can. Speaking for myself, I never really liked making such decisions *when they had the element of a zero-sum game and/or there was a lot of money involved.*
Coming specifically to impact factors, I find the obsession of the chemistry and biology communities with these to be quite ridiculous. I entirely agree with Sachin Shanbag that over-reliance on such quantitative measures is indicative of laziness. To paraphrase the old US Supreme Court definition of obscenity, we recognize quality when we see it, even if we cannot define it. On the flip side, we can also recognize sleaze when we see it, though apparently Nature and its Editors cannot.
Keep up the good work.
Sagar
http://www.youtube.com/watch?v=-VRBWLpYCPY
Post a Comment