All That Matters

As this week there is again a lot of talk about journal impact factors with the release of this year’s data later today, I like to take this timely opportunity to look at citation metrics more broadly, in terms of fundamental flaws in weighing data, and important data missing in the underlying data sets, which in my view miss important data when it comes to practical, technological impact of a study.

I recently had the opportunity of attending a talk by Paul Wouters from Leiden University, a professor of scientometrics. He pointed out one of the fundamental flaws in citation metrics that goes right to the heart of such data collection, before one should even discuss more superficial metrics such as h-index or the impact factor. Like any other piece of data, the context of a citation matters, he said. Factors that play a role are the type of paper where a reference is cited, and in what way. Was it criticism? Controversial papers for a while at least can gather a lot of citations even though eventually their impact on scientific process can be nil. There are also human aspects. Relevant points here are who cited a paper, was it a self-citation, or were there other motivations for citations? After all, citation cartels are not unheard of.

There is a lot of literature on various aspects of citation analysis, and more details on this can be found in Wouters’ doctoral thesis on citation culture, or in the 2008 paper by Jeppe Nicolaisen on citation analysis.

More broadly speaking, I am not sure whether it will be possible to properly analyse and process context when it comes to citation analysis. There are too many ways to game such systems. However, a more complex analysis might well be possible, taking the example of he ranking of web sites in search engines. There, context is everything. A website that is linked from many other sites is not necessarily an important one. Instead, a link to a web site from an important web outlet such as a popular news web site weighs much more than links from unknown web sites. Indeed, many links from news web sites or social networks might also be an indicator of immediacy, further propelling a site up the search engine rankings. […]

Planning electrical grids in a steady environment is not overly difficult. A number of large power stations are connected to urban population centres, where much of the power is consumed. Typically, such power grids would look like meshes with  interconnected distribution points that make sure that if one power station fails, others can compensate .

However, as electrical demand grows, the solution is that new power plants are built and linked to the net at various places, but often with only one connection to the network. These dead ends make the network very susceptible to blackouts, even if many of them are connected by two parallel power lines for redundancy.

In future, the use of renewable energy will pose even greater demands on such network architectures, because the distributed generation of power makes the power generation very dynamic. If the sun shines in certain parts of a country, or the wind blows strongly in one area, large amounts of power will need to be shifted between regions, and the power grids need to be capable of handling that. […]

The Boeing 787 Dreamliner is the most recent major new aircraft design from Boeing, and the manufacturer’s most fuel-efficient plane. I have never had the pleasure of being passenger on one of these, but the design is certainly very modern. Composite materials are widely used in the aircraft, which is key to the plane’s fuel efficiency and explains the popularity of the plane. With more than 800 orders in the books, Boeing was also on a good track to break even commercially.

Then, on January 16 the FAA grounded all 787, following a number of technical problems. Earlier, the Japanese airlines ANA and JAL had already suspended all 787 flights, which was a significant signal because combined these two airlines operate almost half of the 787 delivered to date. Aside from a number of other technical issues such as a fuel leak, a key reason to ground the entire fleet has been two incidences where the back-up batteries overcharged and overheated such that there was the danger of fire on board. But how big a deal are these battery incidents? […]