Sunday, October 12, 2008

I think I have said before that the more I learn about JEE, the more convinced I am that it's a bad exam. The latest detail, which you will find in the graphics accompanying the latest ToI story by Manoj Mitta, is about the marks corresponding to the 80th percentile in each exam. Here are the data for the last three years:

 Year Maths Physics Chemistry 2008 30 21 30 2007 29 30 37 2006 25 24 29

Remember, these marks are out of 160 (or so); in other words, in 2008, someone at the 80th percentile in any of the three would have got a score of less than 20 percent!

Just mull over it a bit: 80th percentile corresponds to a mere 20 percent in each exam.

Now, we also know that the 20th percentile in 2008 corresponded to just 5, 0 and 3 marks in math, physics and chemistry, respectively; this low score was the cut-off used as a first filter. This implies that the middle three quintiles -- 20th to 80th percentile, representing about 180,000 people! -- are squished in the range of 0 to 20 percent!

Now, this sort of stuff would not matter (much) if the exam didn't care about those outside of the top quintile in each exam (Mitta says that using this filter across all three exams would still leave some 24,000 students!). But the IITs are not doing that. For some strange reason, since JEE-2006, the IITs have been using a procedure that plumbs the very crowded middle three quintiles.

When you have a procedure that makes you fish in this very crowded pool, you are not running an entrance exam, you are running a lottery.

[How bad is this lottery-like situation? We know some data on this question, thanks to Charu Sudan Kasturi's report in the Telegraph: practically every sixth rank-holder in JEE-2006 wouldn't be there if the IITs had chosen a different procedure. And in the rank range of 2000 to 5500, it's every fourth candidate who won the lottery that year!]

* * *

I believe the IITs are first rate institutions that insist on using a third rate entrance exam / procedure. Their entrance exam doesn't aspire to any statistical validity, nor to any reproducibility (and we knew that). They invest the rank order produced by such a flawed exam with a sanctity it just does not deserve.

In short, they have let JEE become a war between the IITs and coaching centres. And, in the process, they have taken their eyes off the primary purpose of JEE: to select the right kind of students -- bright, fresh (at the least, not burnt out!), and from diverse backgrounds.

1. Vinod Khare said...

I agree with most of what you said. However, I am still a bit puzzled over the emphasis you laid on the 80th percentile getting only 20% marks.

Now, in a qualifying exam, where the goal is to test how much the student learnt out of what was taught (like the board exam or a college semester exam) this would have been ghastly. One would expect the good guys to retain at least 70 - 80% of what was taught.

However, JEE is a competitive exam. It doesn't matter how much score anyone gets as long as they score better than the others. Isn't it?

2. Sourav said...

I always thought the other way.IITs are a third rate institution which has a first rate entrance exam. The exam evaluation raises many questions,as I can clearly see.However,JEE is still one of the most stimulating intellectual experience for the 18 year old.

and there is no denying that there is a war between the IITs and the coaching (or "cramming") centers.
and that is a war that has to be fought in order to ensure that really bright,intelligent students come out of the system.

3. Anonymous said...

As long as we have this mad rush for IITs, the problem is bound to get worse. Doesn't look like the problem with JEE per se.

Wasn't it just two years ago - the mandarins decided to overhaul JEE pattern radically - ostensibly to lessen the influence of coaching centres? Subjective problems were chucked out, and the paper made more 'objective'. This was done with much fanfare. The outcome - coaching centres still thrive, conduct of JEE is getting murkier year after!

kumar

4. Anonymous said...

Is the situation different for CBSE Medical, AIEEE , CAT , IAS ?

Situation is same everywhere.

In CAT also situation is same.

In fact in IAS also if you look at the results of last 10 years you will find that Toppers get very close to 60 % and all candidate who get around 35-40 % marks pass the examination. In CAT 2007 cut off was 100 out of 300. That is at score 100/300 percentile was 98. at just 33 % score percentile was 98.
and score corrosponding to 20 percentile was close zero.

That means between score 100 and 0 there were 170000+ candidate !! Highest marks obtained were close to 180. thats it.

5. Pilani Pictures said...
This comment has been removed by the author.
6. Pilani Pictures said...

Hi...thats a revelation! But flawed or not the exam is at least transparent.

Being someone who had got thru the UPSC (IAS etc) Exams..i understand the need for transparency in any selection process.

Unfortunately the UPSC has its own plans.. and the aspirants are clueless after the prelims, year after year, abt their scores as well the cutoffs!

The general motto of the candidates is: Take it;forget it!