Sunday, May 26, 2013

DORA: The San Francisco Declaration on Research Assessment

It's good to see some sensible recommendations [pdf] against the misuse and abuse of scientometrics. Lots of professional societies and academies have come together to fight this good fight.

The DORA website has links to extensive commentary from elsewhere. Here's an excerpt form DORA:

A number of themes run through these recommendations:

  • the need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations;

  • the need to assess research on its own merits rather than on the basis of the journal in which the research is published; and

  • the need to capitalize on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).

Another excerpt, this time from Bruce Albert's editorial in Science: Impact Factor Distortions:

The misuse of the journal impact factor is highly destructive, inviting a gaming of the metric that can bias journals against publishing important papers in fields (such as social sciences and ecology) that are much less cited than others (such as biomedicine). And it wastes the time of scientists by overloading highly cited journals such as Science with inappropriate submissions from researchers who are desperate to gain points from their evaluators.

But perhaps the most destructive result of any automated scoring of a researcher's quality is the "me-too science" that it encourages. [...]