Sunday, August 09, 2009

Prathap and Gupta's paper on the research output from India's engineering institutions


Seema Singh's post at Mint Blogs had this eye-catching title: IISc is India's Top Ranking Engineering/Technology Institute, so naturally I wanted to see what all the excitement was about.

The source of this excitement is the paper (pdf), Ranking of Indian engineering and technological institutes for their research performance during 1999–2008, in the latest issue of Current Science by Gangan Prathap (Director, NISCAIR) and B.M. Gupta (NISTADS).

Prathap and Gupta use two different metrics to rank institutions: (a) total number of publications during 1999-2008and (b) a new metric called the p-index, defined as (C2 P)1/3 where C is the number of citations received by the papers during a three-year window immediately after their publication.

Here are the top 10 by p-index:

Rank Institution P p-index
1 IISc, Bangalore 12951 50.17
2 IIT Kanpur 6234 39.27
3 IIT Bombay 7228 36.73
4 IIT Kharagpur 7370 35.37
5 IIT Delhi 6520 32.51
6 Jadavpur University 4807 30.30
7 IIT Madras 5715 29.09
8 IIT Roorkee 3471 25.93
9 Anna University 3687 24.54
10 IIT Guwahati 1596 19.36

I'm not really hot on the ranking part of this exercise; also, I know almost nothing about the p-index (other than the formula), or about why it's better than any other metric for assessing the quality of institutions. So I won't comment on those bits.

But I do want to comment on the validity of comparisons across very many different institutions without taking into account their type, size, age, funding levels, etc:

  1. IISc is an oddball in that list. It's not a predominantly engineering/tech institution. In fact, its science departments are known to have a much stronger publication record than the engineering departments (Giridhar may be able to give some hard numbers to support this point).

    So, a direct comparison between IISc and IITs may not be appropriate, unless one does one of the two things: (a) consider only the engineering departments at IISc, or (b) consider only those publications tagged with "engineering" in the Scopus database. Pratap and Gupta may have done (b), but their paper doesn't say so explicitly.

  2. IISc is an oddball from another viewpoint: it's a post-graduate institution. Almost all the others have strong UG programs, with a correspondingly large teaching load on the faculty. Again, a direct comparison is just not right.

  3. From the formula for the p-index, it appears to favour larger institutions with greater faculty strengths. Thus, IIITs (which are pretty small) compare poorly against larger institutions like IITs. I wonder how a normalized p-index data would alter that list.

  4. While Prathap and Gupta have done a great job in collecting the quantitative data, I don't know what to make of (value-laden) statements like these:

    Although the NITs have been around for a long time (earlier known as RECs), and have been upgraded to deemed university and institutes of national importance status, their research performance is still dismal. In fact, many Indian engineering and technological universities and private institutes are doing comparatively better in terms of performance. ... In addition, the research performance of the IIITs and NITs is disappointing when compared to that of the technological universities and some select engineering colleges. [bold emphasis added]

    But the NITs got the INI status just a few years ago! Until then, they were similar to (at best) the university engineering colleges such as the Guindy Engineering College, Anna University or the Bengal Engineering College (which is now called BESU).

  5. It is inappropriate to compare directly the research performance of an IIT with that, say, of an NIT. This is because NITs (and other engineering colleges) receive much less funding and have much poorer infrastructure.

    In addition (and I think this is even more important than funding), the faculty strength at the NITs (and other engineering colleges) is far smaller than that at the IITs. In this post from 4 years ago, I discussed why faculty strength, by itself, is an important factor. Bottomline: If you are running an UG program and if you don't have at least 20 faculty members, you must be deluded to expect significant research output.

10 Comments:

  1. Vishnu said...

    Quick comment on point number 3. IIT Bombay has nearly 1.5--1.7 times the number of faculty members as IIT Kanpur. So, does it mean that the faculty members at IITK are much more productive than those at IITB? :-)

  2. Abi said...

    So, does it mean that the faculty members at IITK are much more productive than those at IITB? :-)

    @Vishnu: Going by the mighty p-index, the answer is a resounding "Yes!" ;-)

  3. Anonymous said...

    There can be other parameters based on which one can tone down the negative assessment of NITs, IIITs - for e.g. the no. of graduating students continuing to work in core engineering jobs in industry, national labs or getting acceptance to masters/phd programs abroad. Prob. the xtra positive marks to tech. univs and select enggr. college is due to BITS ?

  4. Pratik Ray said...

    The number of faculty is a significant factor for sure. Most departments in NITs would be hard pressed to have 15 faculty members, let alone 20+

  5. Anonymous said...

    The number of faculty in chemistry, physics and maths in IISc is same as that of any major IIT.

  6. Ajay said...

    Just commenting on the NIT's. I have been a bachelor student at one of the NIT's myself. I can say that the NIT I studied in is all about internal politics even among the faculty members themselves. Most of them got their PhD's through the TEQIP program in IISc or one of the IIT's. The worst part of it was that - they never discussed things, came out with ideas, saw the pros and cons of implementing anything. The most powerful guy made all the decisions. With such an attitude - good research would be hard to come their way. I think its more about their attitude rather than funding!

  7. Ajay said...

    Recently an accusation of plagiarism was made on the incumbent director of NITK - Dr. Sandeep Sancheti.

    The accusation was made by the IEEE journal - IET Microwaves, Antennas and Propagation. He was a coauthor of a paper published in 2006-07 and it was found to be very similar to a paper already published in 1990's.

    More details are on my blog at http://donutbekaswami.blogspot.com/2009/08/accusation-of-plagiarism-on-nitk.html

    But yet still - he remains the director and no action has been taken for academic dishonesty. It not only taints his image among the international scientific community but that of the entire university. In the US people do take sincere responsibility and offer resignations! Here no one even seems to care about plagiarism

  8. Anonymous said...

    You have to recognize that research, at-least as far as I know, is performed primarily by graduate students. In India, graduate students are second class citizens, attributable to paucity of research opportunities and interesting professors.

  9. Sharad Jaiswal said...

    As far as the NITs are concerned, its not just the number of faculty, but also the number of PhDs (or some measure of faculty with exposure to research) that is relevant. I graduated from REC Trichy (between 94-98), and not only did the CS Dept. had < 10 faculty, I remember only 3 had PhDs, and a similar number only had BE (and were then doing a Masters).

  10. Anonymous said...

    "From the formula for the p-index, it appears to favour larger institutions with greater faculty strengths. Thus, IIITs (which are pretty small) compare poorly against larger institutions like IITs. I wonder how a normalized p-index data would alter that list."

    Greater faculty strengths means nothing, actually the number of faculties publishing citable papers is more important. Size of the citable papers vs. the uncitable papers. For example, JNCASR, which is a science institution and also which is much smaller compared to these institutions has a p-index of 71. This number is far higher than the numbers reported in the article.