☞ Forecasting Science

As many readers know, I’m interested in trying to understand the regularities behind how scientific and technological progress occurs (The Half-Life of Facts, for example). Part of this process involves the ability to better understand the way in which different research topics are connected and recombined in productive ways: the manner in which knowledge is connected over time as part of the process of generating new ideas and new concepts. This is particularly relevant as our societal scientific outputs balloons exponentially and it becomes more and more difficult to easily understand the entire scientific landscape.

Into this has stepped the Science4cast competition. Building on some previous research around predicting scientific trends—e.g. “Predicting research trends with semantic and neural networks with an application in quantum physics”—this competition is focused on trying to promote work that can identify trends and predict future directions of scientific research:

The competition goal is to capture the evolution of scientific concepts and predict which research topics will emerge in the coming years. We created a semantic network characterizing the content of scientific literature in AI since 1994. The network contains 64,000 nodes, each representing an AI concept. Edges between nodes are drawn when two concepts are investigated together in a scientific paper. Competitors need to predict future states of this exponentially growing semantic network.

The goal is as follows:

…we provide approximately 8,400 snapshots of the growing semantic network – one snapshot for each day from the beginning of 1994 to the end of 2017, and participants are welcome to use more coarse-grained snapshots. The evolution shows how the links between 64,000 nodes are drawn. The precise goal of the task is to predict the future links formed between 2017-2020 in the semantic network, which do not exist yet in 2017. Equivalently, this task asks for the prediction of which scientific pairs of concepts will be investigated by scientists over three years.

I love this approach to science, particularly the use of a human-machine partnership to examine this. This competition also pairs nicely with a new book by Katy Börner called Atlas of Forecasts: Modeling and Mapping Desirable Futures, which, among a great deal of information on modeling and prediction, explores aspects of modeling the directions of science (also check out an earlier book of Börner’s called Atlas of Science).

Want to participate? Check out the competition (and data).


I recently came across this provocative paragraph:

The Guardian article recently quoted an Oxford-educated novelist saying that English degrees are about “what makes us human”. Indeed, the word “humanities” itself telegraphs this claim. But it really is outrageous special pleading. Civil engineering, nursing, sociology, pure mathematics: all of these disciplines and countless others express something beautiful and unique about human nature – our desire to build, the skills of caring for one another in extremis, how we live with one another, our search for the most abstract truths of the universe. I don’t think any of those subjects should feel less central to the human condition than reading Sir Gawain and the Green Knight.

Further links worth checking out:

Until next month.

Loading more posts…