Do read The Telegraph story about the conflicting versions given by the IIT-KGP authorities of how exactly they arrived at the ranking algorithm for JEE-2006; evidently, people who lost out under one of the (not-used) versions are challenging the results of that year's JEE (and they have lost the first stage of their battle in the Kolkata High Court). Here's the key part:
The cutoff marks for math, physics and chemistry — the three subjects tested in the IIT Joint Entrance Examination (JEE) —should have been 7, 4 and 6, respectively, in 2006 using the IITs’ stated formula.
But the institutes, in response to an earlier RTI application, had disclosed to parents that the cutoffs they used for the same subjects were 37, 48 and 55.
Clearing all three subject cutoffs entitles students to be considered for the final cutoff, based on their aggregate scores. Students with the top aggregates among those who have cleared all three subject cutoffs are selected to the IITs.
Thus, it appears that there were two filters: the first selected candidates who met the subject-wise cut-off marks, and the second ranked them based on their aggregates (it's not clear whether it was a simple or weighted sum of the three exam marks; probably the former). In 2006, the first filter used high cut-offs (if they were indeed 37, 48 and 55) which, clearly, would have promoted consistency over variability across the three subjects. In other words, it would pick someone who has scored reasonably high marks in each of the exams (say, 50, 60 and 70), over another who was a star in one subject but scored sub-par marks in another (say, 90, 65, and 45), even though the latter might have had a higher aggregate.
In 2007, IITs played it safe by choosing a very low subject-wise cut-off marks: in the single digits! In this scheme, the variability across the subjects wouldn't matter, as long as the candidate meets the aggregate cut-off.
All this is a great lesson in how one's "rank" depends not only on his/her performance (and luck!) on the day of the exam, but also on how the examining authority chooses to draw its line (over which even the authority does not have any control before the exam, since JEE is not a standardized exam with questions that are 'tested' on a representative population). This clearly puts a fairly big question mark over the sanctity of one's rank -- particularly for ranks beyond 1000.
And it's not clear why the IITs make a fetish out of it.
* * *
Many thanks to Pratik Ray for the pointer.