Friday, August 01, 2008

At a glance: 2008 JEE results


IIT-Roorkee has released a lot of data on JEE-2008. As we have seen earlier, the IITs use two filters before students are given a rank. The first filter gets rid of students who fall in the bottom quintile in *any* of the three subjects (math, physics, chemistry). The subject-wise cut-off marks are, respectively, 5, 0 and 3. Yes, you read it right: in physics, this cut-off is indeed zero.

The IITs have used a reduced cut-off for reserved categories. This concession is 10 percent for OBC candidates, and 40 percent for SC and ST candidates. Thus, for SC and ST candidates, the subject-wise cut-offs were at 3, 0, 1.8 marks in math, physics and chemistry, respectively.

The second filter simply picks the top N students using just one measure (this year, N=7903): the aggregate (or, the raw total of the marks in the three subjects). When multiple people end up with the same total, some procedure is used for breaking the tie. The details of this procedure does not interest me, so I won't go into it.

The cut-off in the aggregate score is 172. The reserved category students enjoyed a concession in the aggregate score as well; thus, the effective cut-offs for the OBC, SC and ST candidates were, respectively, 172, 104 and 104. The reason for the OBC cut-off being the same as the overall cut-off must be because of technicality, which I am yet to figure out. We will re-visit it later, if needed.

In another disclosure, IIT-R has revealed the aggregate scores for a bunch of uniformly spaced ranks. It has given this data for not only the common pool (for which the cut-off is 172), but also the OBC, SC and ST candidates. Let's see what these data look like; in the following plot, the aggregate score is on the horizontal axis, and the rank (in the respective lists) is on the vertical axis. Here's the plot:

Figure 1

The first thing that strikes you is the steep drop-off in the aggregate as you go up in ranks at the higher end. For example, the first five hundred ranks span a range of 146 (287 to 433), the remaining 7403 ranks span a range of just 107! The latter range works out to almost 70 students for each aggregate score between 172 and 287; In other words, there's a whole lot of tie-breaking the IITs would have indulged in from the 500th rank onwards!

Figure 1 is not particularly useful, because our reservation formula ensures that the x percent of the seats in each class (Institution-Department combination) is reserved for the beneficiaries. This means that the top 10 percent of the OBC students (for example) will be studying in the same classes as the top 10 percent of the common pool (and the next 10 percent in OBC will study with the next 10 percent in the common pool). It makes sense, therefore, to use a scaled rank on the vertical axis; thus, the rank in each category is divided by the total number of ranks in that category, so that each curve will end up at 1 at the top/left. Here's the scaled plot:

Figure 2

The striking thing in this figure is the closeness of the OBC curve with that of the common pool. I have to warn you, however, that the common pool includes the OBC candidates. Thus, the curves for the non-OBC general pool and the OBC pool are likely to be separated a little; I am confident that this difference is small, because both the curves end at an aggregate score of 172 (by definition), a cut-off that was used for both the groups.

Here's a second warning: The OBC students form about 14 percent of the common pool (1134 out of 7903). In the next couple of years, this fraction will have to be ramped up (theoretically, to at least 27 percent). If the full OBC reservation had been implemented this year, the additional OBC students would all have come with an aggregate of less than 172. I don't know what kind of cut-off the IITs would have employed for OBCs, but let's assume that they used a cut-off that selected 2000 OBC students. We can now re-plot the above figure and see what effect of a higher OBC reservation would looks like. Here it is:

Figure 3

[Caution: I need to warn you again that this figure is based on speculations about where the IITs might have drawn the line for the OBCs.]

Now, you can see the effect of reservation, which was absent in the second figure. You have the common pool, followed (to its left) by the OBCs, SCs and STs at a difference of about 30 marks, 80 marks and 80 marks, respecitively.

[Aside: One can perform this sort of an analysis for each subject, but that kind of disaggregated data has not been made public.]

Let's go back to the second figure. If we assume (and this is a BIG assumption) that JEE marks are a good predictor of a student's performance in IIT, some conclusions follow. Since the OBC and general pool curves are so close, an OBC student is not particularly at a disadvantage in choosing one of the 'highest' branches available to him / her. This is because the difference between his / her marks and those of his / her classmates are not likely to be terribly large (at least, this year).

Alas, the same cannot be said about SC and ST students. In particular, life would be extremely difficult for someone with a rank of SC 100 (whose aggregate of 167 is already below the common pool's cut-off) to compete against his / her classmates whose ranks are likely to be near 500 (for which the aggregate is 287). Such a student is better off by choosing a "lower" discipline where the students will have scores closer to his / her own.

25 Comments:

  1. cipher said...

    Neat work. I don't dispute most of what you have said. Just one thing though. The difference between OBCs and General category students will always be high for the first 500-600 ranks of the general category and those OBC students who are pitted in the Computer Science, Elec branches will feel the heat. For that matter let me tell you that in first year when all have common courses, those people still stand out. In the case of OBCs they will have to face the same for 4 years.

  2. Pratik . said...

    Just a comment about the scaled plot. Instead of assuming that top 10% OBCs would be in the same branch as the common pool, it is probably a better assumption to group 10% OBC with 60% in the common pool (I assume you meant the general category by "common pool"). Basically, the percentages would be selected as per the ratio of quotas (which of course, would have to be revised over next 2 years).

  3. Anonymous said...

    Such a student is better off by choosing a "lower" discipline where the students will have scores closer to his / her own.

    At least in my time at IIT Madras (2000-04) i used to see this happen regularly. Aerospace and Electrical were supposed to be "tough" branches so SC/ST students would prefer to avoid those and instead take "lower" branches (as defined by choice by general category). in fact, if i'm not wrong, there wasn't a single SC/ST in my batch in Aerospace!

    anyways thanks a lot for the analysis. and thanks to the RTI we can now actually do some good analysis on scores, etc. rather than just speculate as we used to do in our time

  4. புருனோ Bruno said...

    //The reason for the OBC cut-off being the same as the overall cut-off must be because of technicality, which I am yet to figure out//

    How is this possible.


    It is possible that a FC Student with 174 cannot get a seat, but a OBC Student with 174 can get a seat (that is this cut off is only for the call to the counselling, but not an assurance for seats - compare with TN MBBS Cut Offs are assurance for Seats)

    Or

    It is possible when the technicality has been used to give less than the stipulated number of seats to OBC

    If someone files a RTI, it will be interesting.

  5. Anonymous said...

    What is FC?

  6. புருனோ Bruno said...

    http://timesofindia.indiatimes.com/articleshow/3319507.cms

    NEW DELHI: This year, six new Indian Institutes of Technology were brought into being, each with 120 seats — that's a total of 720 seats.

    And yet, because the OBC, SC and ST quotas could not be filled up (as enough applicants could not get the generously-relaxed pass marks set for these categories), as many as 432 seats will go abegging.

    Consider the absurdity of the situation. On the one hand, new IITs are being created at enormous cost; on the other, as many as 432 seats — that's the equivalent of three and a half IITs — are being allowed to go waste.

    In Saturday's edition, we wrote a Times View saying: "To let over 430 seats in IITs go vacant is a criminal waste of infrastructure (such as faculty and physical facilities). Reservations are meant to give disadvantaged sections of society a boost. But where quotas cannot be filled because there aren't enough suitable candidates, the cut-off for general category should be relaxed so that all seats are used up — the cut-off will still be higher than for SC/STs, so no one can argue that it will dilute academic standards. As with airlines seats and hotel rooms, these seats are 'perishable', they must be filled the same year. This should not affect next year's quota."

    This sorry situation is the result of two major education policies framed by the Centre. The government not only commissioned six new IITs, but simultaneously increased the number of quota seats (for which there are simply not enough eligible applicants). Despite the cut-off percentage being lowered in the name of affirmative action, the students have not made the grade. The old IITs can fill some of the seats with students from the preparatory course, but the new IITs have nothing to fall back on.

    IIT-Guwahati director Gautam Barua said that the institute heads who are meeting later this month may ask the HRD ministry to dereserve unfilled quota seats. "There is no time this year but we may try to seek permission to transfer the vacant seats to the general category for next year," said Barua.

    The prospect of empty chairs in the classroom has disheartened faculty members, many of whom echoed Saturday's Times View.

    'Nearly 50% of reserved seats lying vacant'

    Despite Centre's affirmative action by lowering the cutoff percentage for IIT exams, 432 seats still remain unoccupied. "Every IIT seat has the potential to produce a Nandan Nilekani or a Vinod Khosla," said a senior faculty from Kharagpur. "To allow even one seat to go vacant is like crushing a million dreams and aspirations."

    Another professor from IIT-Bombay said that the empty-seat syndrome while not new had been aggravated this year because of the "unthinking way in which the HRD ministry merrily commissioned half-a-dozen new institutes and expanded quota seats without so much as a thought as to whether or not reserved candidates would qualify".

    Down the years, IIT deans have faced the brunt of political interventions. When the first batch of IIT-Delhi students graduated, 47 of the 53 reserved category students failed. The dean was summoned. Recalling the meeting with the "big fat man", education minister Nurul Hassan, P V Indiresan said, "He kept his bulky hand on my shoulder and asked me, 'Professor, yeh kya kar diya?' (Professor what have you done?)" Little has changed. Only two months ago, IIT-Delhi was pulled up by the Minorities Commission for asking 20 reserved students to pack their bags because of "very poor performance".

  7. புருனோ Bruno said...

    Kudos to the IIT Guys... They have done a really brilliant job :) :)

    They have calculated the Cut Off based on the Original (Old) Number of seats

    And

    They have kept the new seats vacant

    and

    thus have "proved" to the world that "Despite the cut-off percentage being lowered in the name of affirmative action, the students have not made the grade. " where as the truth is that the CUT OFF FOR OC as well as FC is the same 172

  8. புருனோ Bruno said...

    Hats off to the champions of Merit in IIT :) :)

    Look at the Media report
    "Despite Centre's affirmative action by lowering the cutoff percentage for IIT exams, 432 seats still remain unoccupied"

    I am not sure that 172 is lower than 172

  9. புருனோ Bruno said...

    //What is FC?//

    FC - Forward Caste
    OBC - Other Backward (BC and MBC who do not belong to Creamy Layer)
    SC - Scheduled Caste
    ST - Scheduled Tribes

    --

    BC - Backward Caste + Scheduled Castes who are not Hindus
    MBC - Most Backward Caste

    -

    OC - Open Competetion in which
    FC
    Creamy Layer BC
    OBC
    SC
    ST
    compete

  10. புருனோ Bruno said...

    I am still not able to understand this issue.

    Are there NO OBC students who have got more than 5, 0 and 3 in Maths, Physics and Chemistry so that the seats are falling vacant.

    That is the QUALIFICATION Cut Off (Am I right)

    The 172 is the Selection Cut Off, which is based on the number of seats. If seats are reduced, it will go up and when seats are increased it has to go down

    Are the IITs are saying that no OBC has qualified by taking 172 as the "Qualyfing cut off" and there by trying to CHEAT the government and public to withdraw the Reservation.

    I may be totally wrong. but if I am wrong, what is the right answer

    Can some one with more knowledge enlighten me

  11. அ. இரவிசங்கர் | A. Ravishankar said...

    This seems like a big scandal and completely insane !!

    The admission committee doesn't seem to know how to implement reservation or they don't want to !

    we don't need complex graphs to understand the scenario !

    This is how reservation works:

    Those who got more than eligibility marks 5, 0 and 3 can all apply to get admitted.

    Say, we have totally 100 seats. and 20 for Open competition, 40 for OBC, 40 For SC-ST (just some random percents)

    Top 20 rankers (including OBC, SC-ST not just FC) will get admitted into the 20 seats for Open competion. The aggregate score of the 20th person will be the cut-off for Open competition. You should note that the cut-off is an incidental score set by the 20th person.

    Then the next 40 seats are filled by the next 40 Top ranking OBC. even if the 40th person got just 5, 0, 3 the minimum eleigible mark, he deserves to get the seat filled. He doesn't need to have a top score such as 172.

    and the procedure continues for SC-ST.

    No one fixes the cut-off first. It gets automatically fixed when the seats are filled. It can only be known after all seats are filled and serves as an indication for next year candidates how much they may need to score to get admitted in their quota.

    fixing the cut-off mark first and driving away prospective students from each quota and proposing to merge these seats to common pool and get some more FC getting added is plain cheating.

    For anyone who follows TNPCEE procedure, we expect that the cut-off marks for FC, OBC, SC-ST should be really distinct and significantly lesser than FC.

    This is how reservation works.

  12. Anonymous said...

    Bruno,

    You may find your answer here -

    //thus, the effective cut-offs for the OBC, SC and ST candidates were, respectively, 172, 104 and 104.//

    Its not only OBC but there are reservations for SC and ST candidates as well with much lowered cut-offs. The report says these seats are not filled.

    All you need is a pair of 'open' eyes to read the statistics.

  13. அ. இரவிசங்கர் | A. Ravishankar said...

    //the cut-off for general category should be relaxed so that all seats are used up — the cut-off will still be higher than for SC/STs, so no one can argue that it will dilute academic standards.//

    as i explained above fixing the cut-off mark first is wrong. even if u fix like that, instead of relaxing the cut-off for general category, y don they relax the cut-off for OBC, SC, ST categories more and bring more students from that section?

  14. புருனோ Bruno said...

    //Its not only OBC but there are reservations for SC and ST candidates as well with much lowered cut-offs. The report says these seats are not filled.//

    Are the IITs are saying that no OBC has qualified by taking 104 as the "Qualyfing cut off" (instead of 5, 0, 3 ) and there by trying to CHEAT the government and public to withdraw the Reservation.

    Can you please answer the question.

    Can you please confirm that candidates from SC, ST who have got more than 5,0,3 have been allotted seats

    //All you need is a pair of 'open' eyes to read the statistics.//
    My Eyes are Open and only since I have kept my eyes open, I am able to find this (?? intentional) anti reservation FRAUD.

    If you can explain to me in a better way or if you can give statistics, i am ready to withdraw my allegations and tender an apology.

    But before that remember that IITs in the past have not been able to even tell the procedure as to how they select students

  15. புருனோ Bruno said...

    It is a testing challenge to the fairness of what is considered one of the most competitive examinations in the world, IIT-JEE. The chief organiser of the 2006 examination, Prof V K Tewari of IIT Kharagpur, is due to appear on Thursday before the Central Information Commission to account for his failure to explain the basis on which he had arrived at subject-wise cut-off marks.

    Tewari has been summoned to show cause why, as "the custodian of the information," he should not be penalized as neither of the versions he gave of the statistical formula tallied with the cut-off marks that had been applied to filter out candidates in the 2006 IIT-JEE.

    In response to an RTI application, the information officer of IIT Kharagpur first said that the cut-off formula was "mean minus standard deviation" for all the candidates who had appeared in the examination.

    But when the figures that emerged from this formula turned out to be at variance with the cut-off marks that had actually been announced, IIT changed its response and said that the performance of only those who obtained at least one mark in each of the subjects had been taken into account. As it happened, even the second version of the formula did not yield the official cut-off marks, which were 37 in mathematics, 48 in physics and 55 in chemistry.

    Worse, because of the vast and unexplained variation in the cut off marks, the less meritorious qualified even within the general category candidates at the expense of those who performed better.

    //LESS MERITORIUS WITHIN GENERAL CATEGORY - A point to note for the champions of Merit //

    IIT Kharagpur was hard pressed to justify the anomalies that came to light: For instance, a candidate who scored an aggregate of 250 marks failed to qualify simply because he got 52 in chemistry (three less than the cut-off marks in that subject) while another qualified with an aggregate of just 156 marks simply because he happened to get 55 in chemistry.

    The summons for Thursday's hearing followed the Calcutta HC's rejection of Tewari's plea to stop CIC from proceeding against him. Information commissioner O P Kejariwal has served a penalty notice on him despite his contention that he has already complied with CIC direction "in letter and spirit" to disclose the cut-off procedure for the 2006 examination.

    Since the 2006 examination was the first IIT-JEE to be held after RTI came into force, IIT Kharagpur's inability to come up with a plausible explanation for the cutoff marks of that year cast a shadow on a much-touted system that had been in existence for over four decades.

    This prompted the seven IITs to adopt such a liberal approach that for the 2007 examination the subject cut-off marks, as reported first in The Times of India , plunged to single-digit figures (1,4 and 3), making a mockery of their purpose of ensuring that selected candidates displayed a certain minimum level of knowledge in every subject.

    In the new procedure, the cut-off marks are pegged to the best marks obtained by the bottom 20% of the candidates in each subject.

  16. புருனோ Bruno said...

    //as i explained above fixing the cut-off mark first is wrong.//

    Ravi, Guess you are not aware of the "policies" of IIT.

    You can only let some one correct the mistake when he wants.

    You can only awaken some one who sleeps
    You cannot awaken some one who pretends to sleep

  17. Anonymous said...

    Neat article. It appears there is a lot of tie-breaks to make. It was always obvious from my JEE days, that was the case. But it begs the question, how the tie breaks are made.

    An range of 100 can never separate 7000 or so people (unless a different score is arrived) by weighting. While the total score is quite good to rank the first 500 people, the fate of the remaining people is less clear.

    It almost seems there must be another level of "effective marks" to rank the rest of the folks or break the inevitable ties.

    Take the simplest algorithm: after a tie break, pick the highest in M and then P. That still only gives a range of 321 (107*3). A larger range may be obtained by changing the weighs for each subject, which I guess is done.

    Does it also mean JEE is an excellent exam to filter the top ~ 500, but perhaps not the rest?

  18. Anonymous said...

    Bruno,

    //remember that IITs in the past have not been able to even tell the procedure as to how they select students//

    In 2006 there was a petition to disclose the procedure but it got dismissed by Calcutta Court. Here are some news about 2008 JEE

    //NEW DELHI: If subject-wise cutoff marks for general candidates in the just-held IIT-JEE are, as in last year’s examination, likely to be in single digits, how will the system have scope to relax the qualifying marks, as announced, by 10% for OBCs and 40% for SCs and STs?

    Chairman of IIT-Jee 2008, Prof N M Bhandari, admitted to TOI that the subject-wise cutoffs for reserved candidates may turn out to be less than one mark which was the level to which the bar was lowered last year in one of the subjects for general candidates.

    Speaking from IIT Roorkee, Bhandari hastened to clarify that since subject-wise cutoff marks for even general candidates have become so low, a further reduction for reserved candidates would be of ‘little significance’.

    The ridiculous cut-offs are thanks to a rather liberal ranking procedure adopted last year by the IIT system, stung as it was by an RTI application seeking statistical basis for the cut-off marks of the 2006 examination.

    In a blow to what is regarded as one of the toughest competitive examinations in the world, neither of the statistical formulae given by IIT Kharagpur to the Central Information Commission tallied with the stated cutoff marks for 2006 — 37 for mathematics, 48 for physics and 55 for chemistry.

    The more transparent procedure adopted in 2007 reduced the corresponding cutoff marks to 1, 4 and 3, making a mockery of their purpose of ensuring that selected candidates displayed a certain minimum level of knowledge in every subject.

    In the new procedure, the cutoff marks are pegged to the best marks obtained by the bottom 20% of the candidates in each subject.

    As a result, 91% of the candidates cleared the cut-off marks, for instance, of chemistry in 2007 as compared to no more than 6% the previous year.

    Since subject-wise cut-off marks have been rendered meaningless in the new procedure, the selection of the candidates, whether general or reserved, now depends entirely on their aggregate marks.

    The number of candidates making it to the merit list will be 1.15 times the total number of seats available in each of the categories.

    NEW DELHI: If subject-wise cutoff marks for general candidates in the just-held IIT-JEE are, as in last year’s examination, likely to be in single digits, how will the system have scope to relax the qualifying marks, as announced, by 10% for OBCs and 40% for SCs and STs?

    Chairman of IIT-Jee 2008, Prof N M Bhandari, admitted to TOI that the subject-wise cutoffs for reserved candidates may turn out to be less than one mark which was the level to which the bar was lowered last year in one of the subjects for general candidates.

    Speaking from IIT Roorkee, Bhandari hastened to clarify that since subject-wise cutoff marks for even general candidates have become so low, a further reduction for reserved candidates would be of ‘little significance’.

    The ridiculous cut-offs are thanks to a rather liberal ranking procedure adopted last year by the IIT system, stung as it was by an RTI application seeking statistical basis for the cut-off marks of the 2006 examination.

    In a blow to what is regarded as one of the toughest competitive examinations in the world, neither of the statistical formulae given by IIT Kharagpur to the Central Information Commission tallied with the stated cutoff marks for 2006 — 37 for mathematics, 48 for physics and 55 for chemistry.

    The more transparent procedure adopted in 2007 reduced the corresponding cutoff marks to 1, 4 and 3, making a mockery of their purpose of ensuring that selected candidates displayed a certain minimum level of knowledge in every subject.

    In the new procedure, the cutoff marks are pegged to the best marks obtained by the bottom 20% of the candidates in each subject.

    As a result, 91% of the candidates cleared the cut-off marks, for instance, of chemistry in 2007 as compared to no more than 6% the previous year.

    Since subject-wise cut-off marks have been rendered meaningless in the new procedure, the selection of the candidates, whether general or reserved, now depends entirely on their aggregate marks.

    The number of candidates making it to the merit list will be 1.15 times the total number of seats available in each of the categories.

    According to Bhandari, it is at this stage that the reserved candidates will get a more substantial benefit as the aggregate cut-off (the aggregate of the last general candidate to have been selected) will be relaxed by 10% for OBCs and 40% for SCs and STs.

    Bhandari is at pains to explain that, though subject-wise cut-off marks are likely to in single digits for everybody, “the emphasis we place on the aggregate marks for ranking candidates ensures that we still get the best in the country".

    All the same, if they persist with the present policy of selecting candidates essentially on aggregate marks, IITs will sooner rather than later have to do away with subject-wise cutoff marks given the farce they have been reduced to.

    //

    Again Bruno,

    Please dont run behind each of these people to see what caste they are belong to as usually you does in your blog.

    The reservation policy came to exist in 2006 and in two years the subject-wise cut-off marks had become a joke. Great going Bruno.

    Nowadays nobody pretends asleep. They are very clear on how to find out short-cut methods to protect their caste-biased desires.

  19. புருனோ Bruno said...

    //In a blow to what is regarded as one of the toughest competitive examinations in the world, neither of the statistical formulae given by IIT Kharagpur to the Central Information Commission tallied with the stated cutoff marks for 2006 — 37 for mathematics, 48 for physics and 55 for chemistry.//

    Well Said. This shows that the method they had used FOR SO LONG, (till they were forced to explain it out) was biased, incorrect and had of course selected less meritorious students

  20. புருனோ Bruno said...

    //The reservation policy came to exist in 2006 and in two years the subject-wise cut-off marks had become a joke. Great going Bruno.//

    I beg your pardon

    The SC / ST Reservation has been in existence for a long time

    The OBC Reservation was introduced in 2006, BUT THE IITs DID NOT FOLLOW THAT. Hope you know this

  21. புருனோ Bruno said...

    //The reservation policy came to exist in 2006 and in two years the subject-wise cut-off marks had become a joke. Great going Bruno.//

    Till 2008, there was NO OBC Reservation.

    Do you agree or Not

  22. புருனோ Bruno said...

    The Cut Off came to single digit. But your statement attributing that to reservation is false, and has bad intentions

    The Cut Off that came to single digit is the GENERAL CATEGORY CUT OFF. Can't you see this simple fact :) :)

    And this has become (or revealed to be single digit) because the IITs were forced to be transparent (so that they cannot cheat and continue their fraudulent ways)

  23. புருனோ Bruno said...

    //The reservation policy came to exist in 2006 and in two years the subject-wise cut-off marks had become a joke. //

    I register my strong protest to the above totally wrong statement which gives a wrong information as if the cut offs have decreased due to reservation.

    Any one who has told the above statement is

    Either a FOOL

    or a

    CHEAT

  24. புருனோ Bruno said...

    Following implementation of Reservation,

    If the Seats available to general category are decreased, the GC cut off should go UP

    If the seats available are same the GC Cut Off should remain the same

  25. புருனோ Bruno said...

    The truth is that IITs have given admission to some one with single digit marks, while denying admission to some one with better marks

    The RTI has brought that fraudulent activity out

    Hence the drop in General Category Cut off

    //Nowadays nobody pretends asleep. They are very clear on how to find out short-cut methods to protect their caste-biased desires.//
    And the one person who had done that is YOU, dear anony. But the fact is that, thanks to RTI, you have been caught :) :) :)