Sunday, June 22, 2008

Do institutions -- and their professors -- have a right to choose?


Alternate title: An auction theory of entrance exams!

* * *

In a thoughtful post, Arunn Narasimhan of IIT-M lays out two possibilities for selecting students (look at this and this for some more background).

While the first mechanism selects students based on high cut-off in each of the three subjects of the exam, the second uses just the aggregate marks. The first method could de-select those -- let's call them specialist stars -- who excel in one subject, but don't meet the cut-off in one or both of the other subjects. A corollary is that the second method may put some -- let's call them all-rounders -- at a disadvantage.

Sidebar

Clearly, at the top end of the achievement spectrum, the difference between two procedures has a very high probability of being small; you cannot be among the top 500 (or, even 1000) unless you score exceedingly well in all the three subjects. But as you go down to ranks beyond 1000, the differences are likely to mount. The Telegraph's story about JEE-2006 shows that this is indeed the case: the number of discrepancies between the two procedures was only nine in the top 1000 ranks, and it increases to 54, 125, 255 and 354 for the next four sets of 1000 ranks each.

To me, the real surprise is the 35 percent difference by the time you come to 5000th rank. Remember, this position falls in the top 0.5 percentile of the pool of higher secondary (Pre-University or Plus Two) students in the science stream! In this exalted regime, this represents a huge divergence between the all-rounders and specialist stars.

* * *

Arunn points out that it is difficult to select one of the two procedures over the other (JEE-2006 went for the all-rounders, while JEE-2007 went for star specialists. But, as I said, the difference is not much at the top end of the spectrum). I don't see why a choice has to be made between these two procedures. Why not select students using both? Since all-rounders and specialist stars bring different sets of aptitudes and attitudes to the table, isn't it a good thing for the IITs?

I would say yes, but only if something else is also taken care of. That something is this: if you select specialist stars, you should find some way by which they get to study fields that play to their strengths. What good does it do to the institution (or, even the student) if someone who excels in math (but scores poorly in the other two subjects) if he/she cannot find a place in computer science, or does not choose mathematics because of parental pressure? Similarly, is it desirable for a chemistry star to skip studying chemistry, chemical engineering and metallurgical engineering, and choose electrical or mechanical engineering?

Let me state categorically again: I am not arguing for a Big Brother who'll 'guide' students to areas that he thinks is right for them. My argument is that institutions -- and their individual departments (and their professors!) -- also have a right to be choosy about the kind of students they want to teach; their exercise of this right need not infringe on the right of individual students to choose what they want to do.

Thus, for example, a computer science department may want a student body that has more than a few specialist math stars (irrespective of how well they did in their physics and chemistry exams). Similarly, a chemical engineering department may want a few chemistry whizkids even though they may not have done all that well in the other two.

If we accept this line of thinking, we still have to address the issue of matching students to specific departments in specific institutions. Currently, the IITs rank students, who then take their turn at choosing an institution-department combination, with professors being mere spectators. Is there an alternative that respects the rights of professors as well as those of the students?

Sidebar

Rahul has argued (see this post and this comment) for dispensing with the idea of assigning a student to to a specific department at an IIT (which is currently being done through a ranking mechanism). Students will then take a common set of courses during the first one or two years, at the end of which they can be assigned to a department; this step will have the backing of the 'revealed competence' of the students (through their grades in the relevant subjects). Let me say that I like Rahul's point about giving students a year or two to explore the intellectual territory before they specialize.

But, in this post, I want to explore a different possibility that allows students to join a specific department in an IIT.

* * *

Let me take a stab at this question:

  1. Let each student get a numerical score (normalized to, say, 1000) in each JEE subject . In principle, every student taking the exam may be given this score; but the IITs may want to not embarrass those with negative marks!

  2. Each department in every IIT announces, before the exam, its preferred weights for individual subjects. For example, a chemical engineering department might choose (0.25, 0.3, 0.55) as weights for (math, physics, chemistry), while a computer science department may use (0.5, 0.3 and 0.2). Heck, another CS department at another IIT may even go for (0.6, 0.4, 0.0)!

  3. While the scores may be sent to as many students as the IITs choose, only those in the top X percentile (say, X = 5) in any of the subjects are invited to give their preferences for specific institution-department combinations. This is best done through a website.

  4. For each department, a list of all eligible students is prepared, in the decreasing order of their composite scores. Similarly, each student may be eligible for multiple fields of studies in different places.

  5. Students are assigned to their top-most choice for which they are eligible.

Such a procedure would not have been possible in the 1980s (for example) when students' choices were processed manually. It can be contemplated now only because the technology for doing it is available. After all, fancy auctions are being used (and some IIT professors are probably designing even fancier ones!) for matching advertisers with search phrases at Google and other online firms; what we are talking about here is essentially an auction -- involving IIT seats as the object of desire and JEE marks as the currency -- that aims to maximize the 'outcome' for the interested parties.

Let me be the first to admit that what I have suggested is probably not the most perfect way matching students to institutions; the procedure I outlined above may not be air-tight, and there may be other mechanisms that achieve the same goal in a more elegant way.

But, what cannot be denied is that individual departments have a right to select their students who, in their opinion (and collective wisdom), will benefit the most from the education they impart. If this right is conceded, I think it's difficult to justify the JEE's current format with its undifferentiated ranking that selects specialist stars but places them in sub-optimal settings.

6 Comments:

  1. parseval said...

    Dr. Abi, a very interesting post!

    Although it must sound trivial, I am obliged to protest on on a minor issue. While it may seem surprising, IMO, chemical engineering actually has very little to with chemistry! (ie, knowledge of mechanisms, pathways, etc that an organic chemist would have).

    IMO, there's much more of physics and math involved in chemical engineering than chemistry :)

    Anyway, regarding the main issue you address, I think that your idea of departments selecting students based on different weighing of the entrance marks is really worth exploring!

  2. pc said...

    If we just stick to the IITs, what is the need for adopting a new system ? Is it the case that students are not performing well under the current system ? And is such a new system really necessary at the under-graduate level ? I can see that giving more choice to faculty at MSc/PhD level would be very desirable. And admission to MSc/PhD is based on GATE not JEE.

    At NIT for example, after the first year, a few changes in branch is allowed based on the students performance in the first year. Does this not exist in the IITs ? Do IITs allow students to move from one IIT to another ?

    I think a system where students get to choose the department after the first year would be unpopular with students. They wont like the uncertainty involved in this.

    In MSc/PhD, students are taken into some department and later the students chooses his adivsor. So faculty do have a little choice even in the present system. All students automatically get scholarship, the faculty does not have to do anything. If the faculty want more choice, they should fund the students from their project money. It may be a good idea to allow both types of admissions, those who get govt scholarship and those who are funded by the faculty projects. For the latter the faculty can have more choice.

  3. chirru said...

    Abi,
    A thought provoking post, a few thoughts/opinions.

    1. JEE ranks, these days, in most cases boils down to how well one's "trained" to crack a vast set of differing types of problems. Its not merely one's acumen for the subject, but also the effort gone into it that influences the score.
    2. Although flair for math is something that comes naturally, I personally believe chemistry is a 'classroom' subject, something that is to be taught. It is easier for someone to score well chem, even if the person doesn't have fondness for the subject.
    3. So, while the line between a good score in JEE math and performance in Computer Science might hold (there are a lot of exceptions here too), the same correlation between JEE chemistry and the depts you mention won't. JEE chemistry (assuming the syllabus is unchanged from 2005, when I gave the exam) hasn't helped me one bit in my dept courses (likewise, for the Chem engg. students, as Parseval notes). None of our core engineering courses have required us to recollect concepts we learned for JEE. If anyone, its people taking up MSc Chemistry at some of the IITs who might benefit. Also, there are people who have done best in JEE chemistry despite loathing the subject, obtained ONE mark above the cut offs for physics/math and opted for CS out of their own volition. This, I think, might be spurred by the fact that a set of CS core courses discuss "algorithms" - solvability of/procedure to solve a problem rather than the solution to it (which requires a strong footing in math), which might be of interest to them.

    In short, good at math => CS engg. is a good option; good at chem !=> Chem/Met engg.

  4. Yogesh K. Upadhyaya said...

    Abi:

    I agree with Mr. Parseval. To say that chemistry is required more for chemical engineering than physics or maths is like saying that biology is required to study medicine, which is not true. Engineering is the subject of applying the basic science for human happyness and all science subjects need same level of understanding to study all engineering branches.

    Different cutoff marks produce different profiles of candidates in final merit list. For example, by keeping 50 marks as cutoff for each subject will produce students with overall 150 marks (50 in each subject) in the merit list, while a student with 249 marks (100, 100, 49) will not be in the list.

    Yogesh K. Upadhyaya
    chemical 1977, IT-BHU
    New Jersey

  5. cipher said...

    A very good post indeed. I was thinking of leaving a comment on the previous cut off posts along similar lines before I lazed out. A few quick points:
    (1) The ambiguity in ranks and selection procedure is not very great till the first 2000 ranks at least(about 3 percent)( Source: Telegraph Story). So the system is not terribly noisy at least till the 1st 2000 ranks.

    (2) The IITs must select a variable number of candidates each year. This number should ensure that the selection is clean. So no matter what cut off scheme is employed a real filtering would yield similar results. Ad hoc filtering just to get a target number of students would always lead to problems of throwing out deserving candidates. The onus lies on IITs to ensure that all 3 subjects have clean distinction between the best and the rest. I remember only one subject being relatively challenging or so on. This should best be avoided.

    (3) Your weighting scheme is innovative and fancy but unfortunately cant be put to much use (according to me). The problem of noise would remain simply because the number of data points to judge a student's capability are very less. Just 3 exams on 1 day.
    A better scheme could be 3 exams + Normalized hi school scores + hobbies and personality in hi school + interviews at IITs in departments of choice + at least 3 semesters of performance at IITs + interviews again before choosing something.
    This will make the students more aware and accountable for the choices they have to make.
    This will also make departments sell themselves if they need to get students of higher caliber and motivation.

    (4) As you have mentioned in your earlier posts, IITs should also start the 4 year undergraduate kind of courses in pure sciences. Many people enter IITs just because they dont have an option of joining a good science college,
    IITs offer very little choice to students and the intellectual excitement in classes is non existent. Due to various reasons students are not motivated and going to classroom is an attendance chore. These could be better ways of cheering students up instead of keeping on trying to tinker the JEE.

  6. Anonymous said...

    Many people come to IIT just for the love of physics and maths. Most are just unaware of their interests. So it really doesnt matter much as to what branch they choose. (There are exceptions tho, but anyway...)

    In the end everyone just ends up doing what the economy offers them. 5 years ago it was software. Today its investment banks and oil companies.

    So it all basically just boils down to IIT degree or not. Branch doesnt matter all that much.