Thed van Leeuwen is a researcher at the Centre for Science and Technology Studies of Leiden University who specializes in bibliometrics: the study of quantitative analysis of academic output. At the ‘Are you ready to publish?’, hosted by TU Delft Library, he lifted the veil off the (in)famous impact factor.

By Aad van de Wijngaart

Scientists never asked for the impact factor. It was introduced in the 1970’s by science policy makers who wanted ‘objective’ criteria for judgment, independent of peer review and local managers. By measuring how often articles were cited, they expected to get a real picture of their ‘impact’ – and that of their authors.

Thed van Leeuwen

The system stuck and grew and now dominates much of academic life. Citations give you credit, and this helps to find tenure and money for further research. So the impact factor is important because it’s been made important.

There’s rather a lot of problems with it though, prompting Van Leeuwen to speak about ‘infamous bibliometric indicators’.

One problem that has complicated citation analysis is personal names that are similar, or are written in various ways. Just think of all the Chinese names that are indistinguishable  in Western writing. Who’s to get credit for what?

More seriously, the impact factor doesn’t measure other scholarly achievements, like teaching, serving on editorial boards or sharing data. And if you happen to work in law or the humanities, you won’t be ascribed much impact at all, since those fields have traditions that don’t fit well in the impact factor model.

Another problem: the impact factor of any scientist may be dramatically different depending on whether you measure it in Web of Science or Google Scholar, which looks at a wider range of publications.

Worst of all, the impact factor favors older, experienced and highly productive authors. It gave us the ‘publish or perish’ culture, but at the same time the odds are stacked against newcomers, irrespective of the quality of their work.

So it’s important to realize that impact is not quality. It only tells you how much influence people assign a colleague by citing his work. Remember that, when you’re looking for relevant sources: never use bibliometrics as a stand-alone tool!

Another thing to take away is how to deal with the impact factor of journals. Don’t worry if you can’t get your article into Nature: Van Leeuwen explained that you can achieve high impact even if you publish in a journal with an average impact factor.