I think I have said before that the more I learn about JEE, the more convinced I am that it's a bad exam. The latest detail, which you will find in the graphics accompanying the latest ToI story by Manoj Mitta, is about the marks corresponding to the 80th percentile in each exam. Here are the data for the last three years:
Remember, these marks are out of 160 (or so); in other words, in 2008, someone at the 80th percentile in any of the three would have got a score of less than 20 percent!
Just mull over it a bit: 80th percentile corresponds to a mere 20 percent in each exam.
Now, we also know that the 20th percentile in 2008 corresponded to just 5, 0 and 3 marks in math, physics and chemistry, respectively; this low score was the cut-off used as a first filter. This implies that the middle three quintiles -- 20th to 80th percentile, representing about 180,000 people! -- are squished in the range of 0 to 20 percent!
Now, this sort of stuff would not matter (much) if the exam didn't care about those outside of the top quintile in each exam (Mitta says that using this filter across all three exams would still leave some 24,000 students!). But the IITs are not doing that. For some strange reason, since JEE-2006, the IITs have been using a procedure that plumbs the very crowded middle three quintiles.
When you have a procedure that makes you fish in this very crowded pool, you are not running an entrance exam, you are running a lottery.
[How bad is this lottery-like situation? We know some data on this question, thanks to Charu Sudan Kasturi's report in the Telegraph: practically every sixth rank-holder in JEE-2006 wouldn't be there if the IITs had chosen a different procedure. And in the rank range of 2000 to 5500, it's every fourth candidate who won the lottery that year!]
* * *
I believe the IITs are first rate institutions that insist on using a third rate entrance exam / procedure. Their entrance exam doesn't aspire to any statistical validity, nor to any reproducibility (and we knew that). They invest the rank order produced by such a flawed exam with a sanctity it just does not deserve.
In short, they have let JEE become a war between the IITs and coaching centres. And, in the process, they have taken their eyes off the primary purpose of JEE: to select the right kind of students -- bright, fresh (at the least, not burnt out!), and from diverse backgrounds.