In a letter in the latest issue of Curent Science, Prof. Ramasesha and Prof. Diptiman Sen (both from IISc), present a strong case against the use of scientometrics in "judging" the scientific contributions of individual scintists.
Scientometry can be helpful in assessing institutions and departments, instead of individuals. This is so because, like statistics, scientometric analysis is helpful when applied to large numbers. It can tell us about the state of activity of groups to make decisions regarding funding and remedial measures to improve a certain institution or department. But, using it to evaluate individual scientists for career advancements and recognitions must be stopped. Henceforth nominations for awards and fellowships, papers for promotion, and application forms for faculty positions should desist from asking for scientometric information of individuals. Assessments must be made solely on the merit of the scientific contributions of the individual concerned.
Ramasesha and Sen are right to demand that application forms "desist from asking for scientometric information of individuals.Here's something that was brought to my attention sometime ago by a commenter: a school of biological sciences at an IIT was proud to display on its website (alas, the URL no longer exists) the h-index expected of its faculty applicants at different levels. The commenter went on to point out that Prof. Venky Ramakrishnan, one of the 2009 Nobel winners in Chemistry, would not be eligible! [Update (10 March 2011): This particular meme is wrong. See this post. See also the comments by Giridhar and Sunil, below. The appearance of this meme here is my mistake, and it doesn't take anything away from Ramasesha and Sen's argument, below, which is a lot more carefully worded.]
Ramasesha and Sen use a similar argument:
Thankfully, for the most prestigious prize in science, it is heartening to see that scientometrics is not the basis of the award. Recent Nobel Prizes have been awarded to scientists who may not rank at the top either on the number of publications or on the h-index. Examples are Venky Ramakrishnan (2009 Nobel Prize in Chemistry)and Koichi Tanaka (2001 Nobel Prize in Chemistry). It is indeed well known that in some instances the work for which a Nobel Prize is awarded becomes highly cited only after the award, as the award highlights the importance of the work.
It's a short, punchy letter. Do read all of it.
6 Comments:
A great post! I really like it!^_^
It's like competitive exams. No one hallucinates they are great indicators of eventual potential in college, but there are way too many applicants for individualized evaluation. Likewise, there are way too many scientists, each too busy writing to have time to read and judge others' work. Using a couple of outlier Nobel cases to discourage the application of standard impact assessment to the vast hordes of mediocre scientists serves no useful purpose. As in airports, stereotyping is unfair, but it is efficient and sort of works.
Prof. Venki Ramakrishnan's h-index is not small (it is 45). His number of citations is more than 8500. If he was an Indian, it would be the highest in the country among biologists. Further, he has published 7, 8, 11 papers in Cell, Nature, Science, respectively. No Indian has this kind of a record.
Sorry to interrupt the scientometric bashing with some actual data.
Thanks
Giridhar
the h-index does have its caveats, but does work pretty well as a ballpark indicator of a scientist's productivity and impact of the work. I'd really like to see some examples of people doing exemplary work who have really low h-indexes, and of people doing mundane, useless work with high h-indexes. And the authors do shoot themselves in the foot by proposing Venky Ramakrishnan as an example. His h-index is really, really high.
@Giridhar:
Since I don't have access to citation databases, could you (or someone else) please check and tell whether the *trends* in Venky's publication and citation records remain similar in other databases as well? Similar to what the MS Academic Search reports for him? I mean, at the MS site, I just found it very interesting that the citation line for Venky was lying very low, close to almost nothing, for as many as *17* years (from 1983 to 2000), and it took a fairly steady upward turn (not a too sharp or too sudden a lift-off), only after the year 2000. Interesting... Does the trend remain similar in other, better, databases as well?
@Sunil:
Ummm... There can be people who are doing mundane, but useful work, and given the government control of science, a few among them *would* be bound to have very high h-indices almost as a matter of syllogism---after all, science also is an organized activity, implying hierarchy of *people*, isn't it? Now, my question is: Why must a mundane work be useless? Most of work in other intellectual, or creative (or even engineering) fields also is only mundane but still useful---and often, the mundane work is highly appreciated (an analogy of having high h-Index). Why should science be any different?
Another point. I don't know what you mean by exemplary, but from what I gather from Louisa Gilder's book on quantum entanglement, I guess that Bell's h-Index would be much greater than Einstein's.
She quotes the following # of citations:
Einstein's SR paper (1905): 700 citations
Einstein's QM paper (1917): 700 citations
Einstein's PhD thesis: 1,500 citations
Einstein's EPR paper: 2,500 citations
Bell's inequality paper: 2,500 citations.
Which work is more exemplary, to you, Sunil? Other readers? Is it Einstein's PhD thesis, or his QM paper? Is it Bell's inequalities paper, or Einstein's SR paper?
With a personal citation record of 0 citations, I guess I don't have any voice here. But I guess I can still point out something. Is it really hard to realize that metrics like h-Index cannot distinguish that kind of a wizardry which takes mankind forward in leaps, from the highly intelligent and yet, rather routine kind of work that still mostly only conforms to the existing concepts and ideas?
And, why are professional scientist so afraid to make any reference to such distinctions? I mean, a similar distinction has been noted by Lee Smolin---he talks in terms of the Talented vs. the Seers. Why do professional scientists, esp. in India, do not talk in such terms? Is there just a bit of the usual turf-defence at work here? Or is there more honest side to it? I am, say, for the want of better words, seriously curious about it.
--Ajit
[E&OE]
Not to bother you all a lot, but I must stand corrected with respect to the comment I just made above---and I can't tell you how excited I am about such corrections!
In my comment above, I had said that I have a citation record of 0 (zero) citations---which turns out to be simply not true! (WOW!)
I do have a citation record, consisting, to the best of my knowledge, of exactly one citation. The citation occurred in an archival __journal__ paper; details below.
A __blog__ post or comment that I made at http//iMechanica.org was cited here:
Guz, A. N. and Rushchitsky, J. J. (2009) "On the problem of evaluation of scientific publications," International Applied Mechanics, vol. 45, no. 3, pp. 233--244, DOI: 10.1007/s10778-009-0188-05
For those who can access the full article---and I cannot---the SpringerLink is: http://www.springerlink.com/content/j667130300555388/ (If you can, please send me a copy at aj175tp AT yahoo DOT co DOT in; thanks in advance.)
I came to know this via a Google search.
And, yes, I am happy to state that my blog post was mundane, but useful. ... Happiness __is__ possible to Man!
Aha, what a feeling! Now, I __do__ have a voice! Hooo! Hoooo!!
--Ajit
PS: @Abi: Please do not moderate this comment of mine, and please do publish it. Who knows, this one may get cited, too! But, yes, happy to let you know that I, too, now have jumped onto get/got-published/cited-in-*journals* bandwagon! (Next time, I will try to observe brevity.)
[E&OE]
Post a Comment