Saturday, January 04, 2014

Navin Kabra's SCIGen Hoax


Consider this: http://iraj.in/Conference/2013/Pune/ICRIEST/

Seems like an impressive conference, and getting a paper published in this conference should be a big achievement on any student’s resume, right?

Wrong.

We submitted to two fake papers to this conference – one was complete gibberish auto-generated by using the online fake paper generator at SCIGen, while the other was auto-generated gibberish interspersed with completely ridiculous statements, movie dialogues, and other random things. Both these papers where accepted by this conference.

More on the saga at his blog post: How I published a fake paper, and why it is the fault of our education system. Mid-day also has a story on Kabra's sting operation. [links via Retraction Watch, and Santosh Sali's comment].

* * *

I have seen far too many cases of poor quality journals and conferences (including at least one scamference) that are ripe for this sort of "sting operation", and I am glad that Kabra has taken to trouble to out someone savvy enough to exploit the demand for such conferences, but hapless enough to get caught in his net.

Sting operations that target conference organizers are entertaining, all right. But, the real curse, which Kabra has written about in the original as well as follow-up posts, is on the demand side -- more specifically, institutional rules that put a gun to people's heads and say "Publish! Or else .."

In the past, these rules put a lot of pressure on faculty members in universities and colleges (irrespective of whether they had access to decent research facilities); the Kabra episode tells us that they have unleashed their corrosive effect on masters and bachelors students as well.

So, basically, there's this huge army of people with a "paper" to submit and/or present; when they are unable to find respectable venues for their papers, someone or the other sets up shop to exploit them. There's a lot of money to be made, after all. And, thanks to our shitty institutional rules, a lot of people become "willing victims" of such scams.

And this is what we are seeing all over the map in India, especially in certain engineering disciplines.

10 Comments:

  1. L said...

    The NAAC gives points for faculty presenting papers at "National/International" conferences. You also get points for organising such conferences. In fact at the exit interview, a member of a NAAC team said that more faculty members must do research and present papers "whatever be the quality" (his words). So all you have to do is tie up with a couple of colleges outside your state. You organise conferences, they send faculty to present "papers". Three months later, reverse roles. So all colleges can notch up points for conferences as well as for the large number of faculty involved in "research" . It works wonderfully .. mutual back-scratching fine-tuned to perfection.

  2. gautam barua said...
    This comment has been removed by the author.
  3. gautam barua said...

    Faculty in "elite" institutions have to be more involved in organising / participating / reviewing papers in tier 2 and tier 3 conferences in India. They should also review papers in journals published from India and in fact take active part in editorial work etc. in such journals. They should publish some of their work in national journals and conferences. Only then will the large number of faculty in colleges and Univs have outlets for publishing their work in "genuine" conferences and journals. Right now the bar of "international" journals is too high for many.
    The proliferation of "online open access" journals where publication only requires a fee, is disturbing. Even more disturbing is that Elsevier, Springer and others are lending their names to these journals (for a fee they are "hosting" these journals). We need open access journals, but who is going to regulate?

  4. Anonymous said...

    Abi,

    Yes, Prof. Barua has a point. (Actually, more than one point.)

    Check out the editorial board of Current Science. And, then, check out its vol. 98, no. 5, pp. 695--697. (I have blogged about it, but don't wish to repeat it.)

    When I was recently doing my PhD at Uni. of Pune, I was concerned with IPR, and that's why didn't want to immediately send publish in journals (wasn't sure if in writing a long paper I wouldn't unwittingly end up giving too many details---and it turns out, some of my ideas were exactly as in LBM (Lattice Boltzmann Method), and so, the caution wasn't exactly uncalled for.) Anyway, that's why I opted for publishing in conferences. And, since I didn't have money and wasn't supported on a fellowship, I could attend only Indian conferences---these were the only ones I could afford out of my father's pension. (JPBTIs in Pune wouldn't give me a job in CAE software development---the broadest category of my PhD research---then, or now.)

    I can tell you from my experience and observations that forget UG/Master's students, even PhD students in India are so desperate to find (good) conferences in which to publish their work. We simply don't have enough conferences and journals being published from India. Going by my informal calculations (of what they told from the conference dais and all), ISTAM conferences then had acceptance rates of 0.5 to 0.33 (2003---2007). Compare that with top international journals: 0.25 to 0.1 (the last figure is for Nature---the most exclusive journal.) And, for the highly touted SIGGRAPH/IEEE conferences in the USA the acceptance rate is 0.75--0.5 is the usual range. I suppose many in your readership would know the acceptance rates of ASME and other journals; no comment necessary.

    On the other hand, returning to my first point, helping, say, Current Science editors is what many readers of this blog could consider seriously.

    More, later.

    --Ajit
    [E&OE]

  5. Anonymous said...

    A correction is due to my reply above.

    I had quoted the SIGGRAPH and IEEE acceptance rates from what I had heard on the UAB campus, but never checked. Either the anecdotal data was inaccurate, or the conferences have improved a great deal. After an informal Google search, their acceptance rates these days (for the conferences held in the USA and all) seem to be 0.35--0.2. Good. (People note these rates even on their CVs.)

    Of course, my broad point still remains. The golden jubilee ISTAM conference (2005), I just checked, had 35 papers printed in its proceedings, out of some 175 papers received for presentations, but no data has been noted as to how many papers were submitted. Even if you assume that every submitted paper was allowed at least presentation (though not publication in proceedings)---which is quite unlikely---it still would give an acceptance ratio of 0.2. The 2006 proceedings says that 165 papers were received, and has 40 of them printed in the proceedings, to give an acceptance rate of 0.24.

    How many NRI folks (esp. JPBTIs having had their summer trainings in top 20 US schools followed by ready VC fundings, but no jobs for someone like me) would be willing to take acceptance rates like 0.2 and 0.24 even half seriously? Esp. if the paper being published came from a non-JPBTI like me?

    Anyway, leaving the personal angle aside, we sure could do with many more (high quality) international conferences in India. Quality does not necessarily deteriorate with increase in quantity---that kind of a static pie thinking is for too socialist/statist in terms of its mentality. If we have more number of _high quality_ conferences, as my informal feel, in India, we should expect the percentage of really high quality papers not to drop any significantly. At least over a few fold increases in the number of conferences. Ditto, for journals.

    And, of course, as I noted in my reply above, we could certainly do also with every possible improvement in the editorial standards.

    Bye for now.

    --Ajit
    [E&OE]

  6. Neelima said...

    The solution is not to throw out the baby with the bath water, viz. remove the publication constraint. Prof. Barua's suggestion that `faculty from elite institutions should get more actively involved in this' makes perfect sense here.

  7. Neelima said...

    The solution is not to throw out the baby with the bath water, viz. remove the publication constraint. Prof. Barua's suggestion that `faculty from elite institutions should get more actively involved in this' makes perfect sense here.

  8. Santosh Sali said...

    Thanks for picking the thread from comment and publishing a blog post.

    One good input here is contribution from good researchers (Gautam Barua's inputs) . Second is actually educating student (And even faculty body) about what is good research. It is surprising that many faculties even with Ph.D. degree can't appreciate a good research. They just don't understand.

    Third important thread will be to generate some institutional norms to improve the quality of research publications and conferences (Guidelines, Norms and then rankings based on it). For example ABDU has a list of journals with their own levels tier-1, tier-2 etc.

    For instance , to give my own example, I wrote a research paper, I was quiet proud of it and wanted to publish in some middle-tier journal. Then I attended a program on hypothesis generation at IIMK , and then I realized how flawed my work is and how poor quality it was. I have dropped now the idea of publishing or even improving that work. And moved on to my next project.

    Point I am making is - there need to be good awareness about what is considered good research in your area of specialization ! Hope the discussion initiates some steps in this direction.

  9. Anonymous said...

    Neelima,

    No, I believe that UG is not a stage to begin publishing papers. (I refer to engg. education.) Please see point no. 12 in my post here: http://ajitjadhav.wordpress.com/2011/05/24/answering-the-questions-for-prospective-faculty-at-iisc/

    Allow me to elaborate.

    Broadly speaking, let's keep in perspective the demands that can rationally be made at each of the UG/Master's/PhD and Post-Doc levels.

    In UG projects, you mainly learn to do literature search, and possibly help execute some experimental procedures or help in computational modeling or theoretical work---but only as a help, under someone else's direct supervision. And, I need not tell you how difficult and important a skill lit. search is.

    In Masters' projects, you learn to formulate one research idea and execute it---essentially, write one paper, or do equivalent work, whether it is published or not. The idea even may not be your own, only of someone from your advisor's lab (say his PhD student). But you learn how to independently pursue the idea, develop its implications/sub-threads to a stage of maturity as to possibly result in one paper.

    In PhD thesis, ideally, you learn to formulate your own themed research program (of the scope of about 3 journal papers). Even if it be aligned with the broad interests of your faculty, ideally, you formulate a theme or the thesis---independently yourself, be a world-class expert about its immediate research area, publish papers and also are able to defend that thesis to your peers (i.e. established independently working researchers).

    In Post-Doc, you learn to write research proposals and actually getting funding, so that, after this stage, you can be an independent PI.

    If someone is brilliant enough at any given stage to be able to master also the next stage(s), he must be allowed that freedom. But this does not mean that it should be demand. And, by "brilliant" I mean: really brilliant. Prof. Agarwal's UG students at IIT Kanpur is a well-highlighted example. But let such exceptions remain only exceptions. For the rule itself, if a UG student learns literature review skills, that by itself is a great deal. (I myself only began doing that during my UG---of course, lib. facilities were limited in those times.) And, by literature review, I also mean other similar skills such as ability to consult design stanadards, product manuals, etc., and make informed choices. That's enough for a UG level. (If anything, I would say: your review should span over at least 30--40 well-selected papers, out of which you read and are able to summarize at least 5--10 papers. Or, equivalent. At UG level, that's enough.)

    So, if there is any such constraint, it should be removed for UG.

    Please also see my next comment, reply to Santosh.

    --Ajit
    [E&OE]

  10. Anonymous said...

    Santosh,

    You raise good points. (BTW, what is ABDU?)

    Further to what you say (educating students and even faculty) about what is good research, I think one good area that is quite neglected in India is: the ritual of the journal club.

    I call it the ritual because I want to see it being made a religiously followed activity.

    In contrast to the research atmosphere in the USA, the idea of journal clubs is starkly absent in India---it was sporadic even in IITs in earlier times; I don't know how things go today.

    Every one knows how the requirement of a "seminar" is handled at the UG level. (Why, there are UG project reports available for buying, and often, engg. school professors themselves are the suppliers.)

    If institutions adopt the requirement to follow a weekly journal club, right at the UG level, and make it mandatory to publicly blog about it, things might improve. (If the requirement is only to send reports to college/university authorities, all of us know how it is going to end up.)

    Just two cents.

    --Ajit
    [E&OE]