The latest issue of Current Science carries two responses [pdf] to the letter by my colleagues Prof. Diptiman Sen and Prof. S. Ramasesha on the inappropriate use of scientometric data to evaluate the contributions of individual scientists. Interestingly, both the responses are from IISc colleagues: Prof. Gautam Desireaju and Prof. M. Giridhar; they are worth reading in full , not least because they are friendly to people with a short attention span -- together, they occupy just one printed page.
This bit from Prof. Desiraju's letter is priceless:
At the lowest levels, h-indices are useful as criteria of eligibility and basic competence. They should be used as criteria of elimination and not as criteria of selection. They will always serve a purpose in Indian science, because it was possible, in the days before we had scientometric indicators, for committees of wise men to simply declare an incompetent as an outstanding scientist. This is no longer possible. [Bold emphasis added]
3 Comments:
How do you address the fact that there are large differences in h-index/scientometric statistics across fields? At the two extremes are mathematics and experimental fields. The average citations per paper for different fields is given here in Table 1 of this paper: http://arxiv.org/ftp/physics/papers/0607/0607224.pdf (paper proposes scaling the indices, but that may not improve the significance of citations to the field)
Firstly, incompetent scientists judged by committees as outstanding, can still fudge their h-indices by writing several review papers. There's nothing in its computation that makes the h-index (or other commonly used statistics) hard to manipulate. (e.g. the classic case of M. El Naschie- a committee may well judge the infamous M. El Naschie as outstanding, with scientometrics failing to catch the fraud.)
Let's say you use scientometrics for assessing scientists, there are four possible states- true positives, true negatives, false positives, and false negatives. Using it as an elimination criterion as in the example above, can still give rise to the last two of the errors, you judge someone as outstanding, but the scientist is incompetent by scientometrics, or outstanding in spite of bad scientometrics. An extreme example is Grigory Perelman who has the outstanding accomplishment of proving Poincare conjecture to his credit, and yet has an h-index of just 5, or Andrew Wiles with an h-index of 12, Michael Atiyah with 10, and so on (these are not terrible h-indices, but certainly don't reflect the genius.) Were one to judge them in any way- elimination/not, based on scientometrics before their best work made them famous, one might have judged unfairly, some truly creative scientists.
I apologize for the mistake about Michael Atiyah. Searching ISI Web of Science for "atiyah mf" yields an h-index of 33.
@Abi: Re. the priceless bit.
I shall restate as many times as is necessary (perhaps till the time that the restatements are taken to be recitations by Indians at large or until you begin moderating my comments out, whichever earlier):
(i) that I was not given admission to IISc as stated in my earlier comments (go, use Google);
(ii) that I have made claims in my official PhD thesis of having new results/perspectives first in ~75, ~187, and ~200(+) years (again, go, use Google);
(iii) that I have h-Index of 1 (or perhaps infinite, if you insist on having only *journal* articles in the denominator too) but not one respectable enough to escape elimination (and what number would you say acceptable, Desiraju---2? 3? 5? 10?)
(iv) that IISc professors refuse to either trust or verify my claims (once again, go, use Google);
Yes, I can be as reciprocally ashamed of any scientist(s) or his master(s), esp. the Indian ones, as the situation requires.
In this instance, go, figure, Desiraju, Abi, the priceless basis on which you would want to eliminate me (from some unnamed process of selection to whatever post related to *science/engg* you have in mind and communicate presumably telepathically to each other)!
--Ajit
[E&OE]
Post a Comment