Tuesday, June 17, 2008

Some more on JEE cut-offs


Another title for this post: Some more on why JEE sucks.

Manoj Mitta of ToI does a great job of following up on his previous reports on the cut-off marks in JEE.

In his previous report, we found that the subject-wise cut-offs in JEE-2007 were as low as 1, 4 and 7 in math, physics and chemistry, respectively -- these were the marks scored by someone at the 20th percentile of the JEE takers in each exam. At that time, several commenters pointed out that meeting the cut-offs doesn't ensure that you will get a rank. At that time, I too wondered why the IITs chose such low cut-offs; if they are into selecting top students, the cut-offs should perhaps be at the 80th percentile, no?

With further revelations about the inner workings of JEE, it became clear that the IITs followed a two-stage process, with the first stage (based on the subject-wise cut-offs) short listed candidates, followed by the second stage which rank ordered them according to their total marks (I sitll don't know if this total is a simple sum or a weighted sum of the three marks).

But we still didn't have the answer to the key question: is it possible for someone with low marks in one of the subjects to still get a JEE rank (because of a stellar performance in one or both of the other subjects)? Thanks to Mitta's latest report, we now know the answer: yes, it is possible. Here's an example cited by Mitta:

Consider the case of the candidate who scored just 12 [or, 7 percent] in mathematics, a subject crucial to all engineering branches, and yet managed to get rank 3,989 on the strength of his aggregate of 239 marks.

Another candidate who got merely 12 more in mathematics and 6 more in the aggregate, however, jumped 590 rungs higher in the AIR list: his rank 3,399 was, according to the latest counselling browser published by IITs, good enough to secure admission in 2007 in IIT-Kanpur and IIT-Kharagpur.

Mitta opens his article with an understandably scathing remark:

... [I]f you score 7% in your Class XII mathematics paper, you fail. But if you score 7% in your IIT-JEE mathematics paper, you can still make it. That's exactly what happened in the 2007 entrance exam

Thanks to RTI, we now know more about the real state of JEE and its ranking procedure: they are a mess. Yogesh had a perceptive comment about it sometime ago:

IIT-JEE is supposed to test the logic of students, not that of examiners.

14 Comments:

  1. Rahul Siddharthan said...

    It is not surprising to me that a "marks percentage" of 7% may get you into the IITs -- such a percentage indicates nothing in the absence of other information about the examination. If you ask me to answer 10000 utterly trivial questions in 1 hour, I will probably obtain 7% (assuming I can fill in each answer within 5 seconds, not needing to think at all). If you asked me 10 questions, each of which I could do in one hour, I would obtain 10%.

    The question is, why is the JEE so difficult that even someone who ranks in the top 5000 obtains only 7%? The answer must be the obsessive need to rank a few thousand candidates individually. In a "normal" examination, they would all score 90% or more, so in a 300 mark examination there would be 30 marks distinguishing them, and perhaps 200 "top rankers". Won't work for the IITs, who must allow their top candidates to choose, say, computer science in IIT Kanpur even before they join. And even so, I don't see how it does work. If an aggregate increase of 6 marks results in a change of 590 rungs (nearly 100 rungs a mark), how do you distinguish candidates separated by one mark? And is it in any sense fair to do so?

  2. Pratik . said...
    This comment has been removed by the author.
  3. Pratik . said...

    If you score 7% in your class XII exam, you would fail; consequently you would not even be considered for admissions to IIT :P

    But Mitta's point is accepted: 7% is too low for cut off to IITs. As you say, IIT JEE rankings right now is a sham. I wonder though, prior to 2k5, was the same process followed for rankings? In other words, when IITs failed to explain the higher cut off marks, did they bend over too much on the other side and lowered the cut offs excessively?

  4. Anonymous said...

    Mitta's point accepted , Mr. Pratik? . What a crap ! So IIT cutoff's should be 90%. Great ! Mitta is an ass who does not give data, nor the basis of ranking - He does not have it. When such rankings are conducted, there will be anomalies pointed out for rank 3399 (GAWD !). Can you point out discrepancies in rank 1-1000? Why not? Because of what the first commentator points out. Mr. Nanopolitan, you need to first go thru a basic course on statistics before giving crap. Justify to me with examples how the JEE marking is biased. And prove it then with data. Forget Mr. Mitta, look at the responses that are there for his article on TOI.

  5. Niket said...

    Abi,

    ANY METHOD that tries to rank top 1% of the students taking an exam is going to face the same issues. As someone (Vivek of vivekspace?) pointed out sometime back, you are being a bit unfair in your criticism of the JEE.

    The problem will stay no matter what mechanism you use to admit 5K students to IIT and rank them as well.

    And no matter what method you use, people with resources to spare on training and coaching will still overwhelmingly outnumber the others who aren't as fortunate, aptitude and skilss notwithstanding

  6. Niket said...

    While I don't care too much about JEE myself, I am more worried about the fact that more and more students these days are asking questions during their counselling (to select institute and branch of interest for their UG) whether such-and-such will be a "cool course" to pass easily and concentrate on MBA.

    This student has just got a JEE rank and has already started worrying about getting admission to an MBA programme.

    And these are sensible people; I am a lunatic to think something is wrong in how we value education.

  7. Rahul Siddharthan said...

    Anonymous: note carefully that I do not approve of this need to rankindividually.

    Niket: the question is, why rank them individually? I think it's extremely unhealthy. A good university would admit all students, give them essentially the same coursework for about 2 years, and allow them to specialise only in the third year. This would reduce the stress of the JEE and would allow students to explore truly interesting new fields, rather than all aim for the "hot" courses.

  8. Anonymous said...

    Dear Rahul,

    Your comment to earlier anonymous is skirting the issue. Your suggestion that they do general courses for 2 years marks your ignorance of how the IIT education has changed for each branch - earlier most courses were similar for 2 years, no longer today, since specialization is the order of the day. And letting students choose specializations will only result in mayhem. Who will not want comp sc ? Maybe only a handful of 300 students in IIT/K. So what do you do? Back to screening. It does not matter what tool you use and JEE ranks are a pretty good screening device. You can shout and 'disapprove' of anything, but you cannot deny that people with higher ranks in the IIT are in general sharper than people with lower ranks. Overwhelmingly so. Also JEE ranks are a very good predictor of final GPAs at IITs. What are you cribbing about? And yes, you and Mr. Mitta still need to present an algorithm which will give us 5000 ranks with a total of 300 marks. Easy, very easy to criticize JEE - which has stood the test of time - and very difficult to give sane suggestions and workable solutions.

  9. Pratik . said...

    Anon, Can you please point out where I mentioned the figure of 90%? In case it is *data* that you want, go through the articles carefully as well as the earlier posts on this blog, and you will find that 7% corresponds to the 80th percentile. When you are going to select the top 2% students, why do you want a cut off corresponding to 80th percentile?

    Some more data for you, Mr Anon, to ponder on, over here.

    You asked for 1 anomaly in 1-1000. Check the link, and you will see there are 9. Heck there is one inside 500 itself, and a total of 994! 994 out of 5000 seats is about 20%.

    What say anon? An error of 20% is statistically insignificant eh ;)

  10. Rahul Basu said...

    I know little about how JEE is conducted and these posts and comments have done little to illuminate the issue. What I find
    curious though is that almost all B.Tech's I have interacted with are substantially superior in ability, performance and general aptitude than almost any other students that I have known, including those we select through our own JEST system. It may be fashionable to berate the JEE system and no doubt like any other exam systems it has its flaws, but surely they are doing something right, despite the 7% cutoff.

  11. Rahul Siddharthan said...

    Rahul B: A few hundred thousand of the brightest high-school students take the IIT-JEE each year, and only the top 2 percent (by some metric) get in. Regardless of how flawed the metric is, I would certainly hope that it selects talented students. That is not the point at all.

  12. Anonymous said...

    What percentages are you talking about? Do these calculations take into account that, of-late JEE is operating with negative marking? Or is Mr.Mitta being very loyal to the 5th standard Maths teacher, who taught us how to calculate percentages?

  13. Anonymous said...
    This comment has been removed by a blog administrator.
  14. Ninad said...

    There is a lot of confusion among common people on this topic...
    Let me explain to clear JEE u have 2 clear cut-off's 1 subject-wise and 1 overall .. the overall cut-off is the minimum score u need to secure a rank a valid JEE rank (upto ~ 8000 outof 3 lac) and u need approx 4k rank to get into engg branches..Then second type of cut-off's are subject cut-off(different for each subject) which in past used be bit more , but resulted many guys failing to get into main list..eg. suppose main list close at 300 at subject cut-off at p-80,C-90-M-70 , and my frnd got p-140,c-150 and m-69 ,then his total will be 359 way above 300 and he can get air ~ 3k,get into -> decent IIT BUT maths 69 insure that HE WILL NOT QUALIFY JEE,lot of student lost their dream coz of this , hence JEE admission commity reduced subject-wise cut-off to top 80% Now therefore subjectwise cut-off become negligible and hence one will not loose his chance to study in IIT coz he's weak in chemistry ....but looking at no candidates appearing I think top 10% cut-off will be fine.
    In simple words your total performance will now matter rather than individual performance in subject.Bcoz we have now combined paper instead of individual subject paper in JEE
    PS- none of the other exams (AIEE/BITSAT) hav subjectwise cut-off ,put it in layman's language technically u can get into BITS/NIT with 0%in maths