All That Matters

I’m glad I’m travelling this week, as yesterday the impact factors have been announced. I actually forgot about the pending announcement yesterday, when some of my hosts here in Singapore reminded me about it at dinner(!). And sure enough, last night twitter was all abuzz with impact factors.

The reason I am happy I’m away from all the buzz is that the impact factors tend to get quite overemphasized. I mean, really, what does it mean, ‘impact’ factor. Per definition, the 2010 impact factor counts the citations in 2010 to papers published in 2008 and 2009, divided by the number of those papers. For more details, there is an explainer by Thomson Reuters, who publish the Impact Factors.

So what kind of impact does it measure? That of papers published there? Well, citations to papers vary a lot. Take my journal, Nature Materials. The most cited paper from 2008 for example is a review on Biosensing with plasmonic nanosensors that has 473 citations so far – coincidentally, that’s a review that I commissioned, and sure I’m glad I did. Going further down the ranks of the most cited papers from 2008, the paper at 10th place got 175 citations so far, that at 20th place 104. In other words – the impact factor has not much to do with the quality of a single paper, the distribution of citations varies a lot.

Therefore, the impact factor certainly doesn’t measure the ‘impact’ of papers, and by implication nor does it measure that of researchers. It provides an average number for a journal. But that even that doesn’t look like an absolute measure to me either. The impact factor of Nature Materials is 29.897 (yes, it’s calculated with such silly accuracy). In comparison, that of our sister journal Nature Physics is 18.423.  So does that mean that Nature Materials is 62% better than Nature Physics? You better start reading the journals, as that’s certainly not the case. Of course, a journal like Nature Materials might be perceived to publish papers on average better than some journals with an impact factor of say below 10. But as a researcher you would have already known that from reading the papers published in a journal, wouldn’t you?

So what use is the impact factor number? Well, being cynical one could say it is a quick measure for those that don’t read the journals but still want to know how good they are on average. The danger is of course that this is then used as a kind of metric to assess the quality of research or to decide on the career of researchers. As it’s clear from the examples above, it certainly should not be used for that purpose.There are better ways to judge the merits of published research, such as article-based metrics and not journal-based ones. Not even my salary as an editor depends in any way on the impact factor of the journal I edit, so certainly it shouldn’t impact those of researchers.

And that’s also because the impact factor is a woefully short-term metric. On Monday at the conference here in Singapore I listened to an interesting talk by Jonathan Adams from Thomson Reuters, and he showed a citation statistics whereby for most disciplines in the physical sciences the number of citations to papers steadily increases over the years until it reaches a maximum at around 12 years (give or take a few years). So even on average for all publications considered, measuring citations for only the last two recent years can mask the true impact of a paper.

Where such short-term metrics can be useful, however, is as evidence for considerable editorial efforts by a journal. For example in case of the remarkable 30% increase in impact factor for Nature Materials’ competitor, Advanced Materials, whose impact factor now stands at 10.857. My congratulations to them on their hard work! But all in all, we shouldn’t overplay the relevance of impact factors.

Following the letters of the law can sometimes lead to seemingly absurd situations. A university in Germany elects a new rector. Two months later the science ministry rejects her appointment, because she had previously cancer, and the possibility of a recurrence is too high for the government to accept. No, she doesn’t have cancer at present, but the mere possibility seemed enough to prevent her from taking her office.

Sounds incredible? Well, that’s exactly what happened to Renate Lieckfeldt (link in German), who got elected rector of the University of Leipzig, a college in the German state of Saxony end of last January.

The dilemma is that Lieckfeldt is a professor at a college in a different German State. And the law is that she has to become a civil servant of Saxony to assume her new position. In Germany, being a civil servant means that you have sponsored access to expensive medical healthcare, typically with much better privileges than those who are on general healthcare. All the way into retirement! And what at least I wasn’t aware is that apparently if you are in danger of costing the state a lot of money because of a medical condition it can mean you are denied becoming a civil servant. And that’s exactly what is the situation now.

Of course, this law isn’t only for university rectors, it holds for every civil servant there. Therefore, my feeling is that the state wants to avoid a costly precedent. So that’s what they follow the law without exception, even though I am sure there would be room for maneuver.

Still, looking beyond Lieckfeldt, what I find outraging here is that a government takes decisions on hypothetical medical conditions. Where would you stop this? Would you stop employing overweight people for the danger of them becoming diabetic? What if you have a gene that has you predisposed for certain diseases? Genetic profiling is widely criticized particularly in Germany. Yet, taking such a decision based on potential health implications such as that following prior cancer is little different in its consequences.

Governments have to lead by example.  There are statistical possibilities lurking in all our genes. Stigmatizing the healthy for past and possible future diseases therefore seems just plainly wrong to me. The State of Saxony in my eyes would be wise to chose the first choice for the job – the already elected candidate.

The crisis at the Fukushima Daiichi nuclear plants is a real tragedy. Tens of thousands of people have been evacuated around the plants, many of which continue to live in shelters with little comfort and privacy. And even worse, there are more than 27,000 people that are either dead or declared missing as a consequence of the earthquake and the tsunami.

The stream of media reporting on the status of the Fukushima plants is continuing, although ironically we are now in a situation where although the continuing release of radiation into the plant’s immediate environment is accumulating to radiation levels that are worryingly high, the broader interest on the issue outside of Japan appears to have ebbed away. And that despite the fact that these problems will be with us for months, if not years.

What is still going strong in the media, however, is the debate on the future of nuclear energy. Some see the accident as a sign that we should stop all nuclear power plants – immediately – whereas others such as George Monbiot see the fact that the implications of this accident so far seem geographically limited as a sign to support nuclear power. Unfortunately, this pro/contra nuclear is where the debate stops, and there appears little movement on either side.

It’s about our energy future

What I am missing in this entire debate is the vision for our energy future. That’s because a sustainable energy supply is a complex issue, where broad brush strokes such as pro or contra nuclear unfortunately don’t help. Take the German government’s decision to shut down seven of its oldest nuclear reactors: unlike the shutdown of nuclear reactors in Japan this hasn’t led to power cuts in Germany. So where does the missing energy come from? This power is bought on the international market. So who can offer spare capacities of around seven gigawatts power or more? My guess is that most likely it’s nuclear energy from elsewhere….

But short-term politics and Fukushima-related knee-jerk reactions aside, how do we envision our energy future? […]