Sunday, August 26, 2012

"Impact factors" and the damage they do.

The Wall Street Journal ran this article yesterday, which basically talks about how some journals try to manipulate their impact factors by, for example, insisting that authors of submitted manuscripts add references that point to that specific journal.  Impact factor is (according to that article's version of the Thompson/ISI definition) the number of citations of a journal in a given year, divided by the number of papers published in that journal over the last two years.  I've written before about why I think impact factors are misleading:  they're a metric deeply based on outliers rather than typical performance.  Nature and Science (for example) have high impact factors not really because their typical paper is much more highly cited than, e.g., Phys Rev B's.  Rather, they have higher impact factors because the probability that a paper in Nature or Science is one of those that gets thousands of citations is higher than in Phys Rev B.   The kind of behavior reported in the article is the analog of an author self-citing like crazy in an effort to boost citation count or h-index. (Ignoring how easy it is to detect and make corrections,) Self-citation can't make a lousy, unproductive researcher look like a powerhouse, but it can make a marginal researcher look less marginal.  Likewise, spiking citations for journals can't make a lousy journal suddenly have an IF of ten, but they can make a journal's IF look like 1 instead of 0.3, and some people actually care about this.

Why should any publishing scientific researcher spend any time thinking about this?  All you have to do is look at the comments on that article, and you'll see.  Behavior and practices that damage the integrity of the scientific publishing enterprise have the potential to do grave harm to the (already precarious) standing of science.  If the average person thinks that scientific research is a rigged game, full of corrupt scientists looking only to further their own image and careers, and that scientific research is no more objective than people arguing about untestable opinions, that's tragic. 



No comments:

Post a Comment

Please, "Sign Up" Here and Get Free Safety Module