How to build more useful mathematical models for society

Mathematical models are receiving an unprecedented amount of attention as a result of the COVID-19 pandemic. Attention that is, perhaps, not entirely welcomed by the scientists in the modelling community if the recent article, Five ways to ensure that models serve society: a manifesto, published in the 25 June issue of the journal, Nature, is anything to go by. The article was written by a group of 22 scientists, headed up by sensitivity analysis expert and Open Evidence researcher, a spin-off of the Universitat Oberta de Catalunya (UOC), Andrea Saltelli.

Uncertainty and political polarization

The manifesto highlights the fact that these models are being used to provide answers on issues such as how the pandemic will evolve or the effectiveness of the measures being implemented when much of the relevant fundamental data required has yet to be established, for example, those relating to the prevalence and fatality and reproduction rates of the virus, to name just three of the many possible variables. Any slight variation in any of these figures has the potential to provide completely different results and predictions.

Concern also exists in relation to the frequent political use of these results to support positions that are supposedly backed up by science when, in fact, that is not the case and the hypotheses in question are based more on the unknown than the known. As Andrea Saltelli explained as he outlined the circumstances that prompted him to put this manifesto forward, "I was afraid that modelling might polarize political debate towards that which was easily quantifiable, neglecting those aspects that are less well quantifiable and the uncertainty that affects many others."

Five points to keep in mind

Nobody doubts the usefulness of mathematical models, much less those responsible for their application, but they do acknowledge the need for them to be used with humility and transparency, with an awareness of their limitations and a commitment to preventing their manipulation in support of hidden interests. The five principles set out in the article for meeting these objectives and ensuring that the models are truly useful to society are as follows:

1. Mind the assumptions: Because models are frequently used that were originally developed for one specific situation and are not applicable to other scenarios. Or because we are not in possession of the relevant data needed to run them. Global uncertainty and sensitivity analyses can mitigate these effects.

2. Mind the hubris: Overly complex models can become problematic and may even end up being useless or much less precise than other simpler and more effective options. There is a great temptation to keep adding new parameters but it is important to remember that this also increases uncertainty and the potential for cascade error.

3. Mind the framing: The models, at least in part, reflect the interests, disciplinary orientations and biases of the developers. The techniques and tools that are chosen are not always neutral and end up influencing the results. Transparency is essential in this case to avoid, among other things, hidden economic or political interests, as well as to develop clear guidelines with which to develop new models.

4. Mind the consequences: Quantification can be counterproductive and an obsession with numbers may end up obscuring other significant explanations, generating a false sense of certainty or mixing, as in the case of the current pandemic, easily quantifiable realities - such as the number of available hospital beds in an area - with others that are not so simple to quantify, such as, for example, certain aspects related to fundamental rights or civil liberties.

5. Mind the unknowns: It is important to acknowledge the infinite number of factors that are unknown. That is the most intelligent approach to adopt with regard to the potential deficiencies that will inevitably be encountered in the development of any model.

A before and after for science

What is clear is that we are facing a significant challenge, perhaps the greatest in decades and one which could mark a before and after for mathematical models and the way in which they are developed. Saltelli goes a step further, however, both with regard to the potential risks and the opportunities generated by the pandemic: "It is more likely to be a turning point for science in general, which is very dangerous because it could either lead to a collapse of trust or to a new covenant being established between science, mathematical models included, and society."

The article also makes the assertion that good modelling cannot be done by modellers alone but is a job that, like so many others at the forefront of this pandemic, is more of a social activity, in which we all form part, in one way or another, and have a role to play. The inevitable question would therefore seem to be, how can citizens contribute and help with this task? Saltelli's emphatic reply: "By looking at the numbers with healthy circumspection, just as we should with any scientific result." He immediately followed up by saying that the public should also be aware of the fact that there is always some other model waiting in the wings, ready to further relativize the conclusions.

The manifesto: its supporters and critics

Following its publication in Nature, the article generated around 1,000 tweets of support, as well as attracting a certain amount of criticism from a few professionals within the sector, although, in the words of Saltelli: "I was expecting many more because we say that complex models are not necessarily better and a lot of modellers are committed to doing just that, building bigger, more complex models which, in the end, become so big and slow to execute that it makes their properties impossible to study."

^{Reference article}

^{Saltelli, A., Bammer, G., Bruno I. et al. (2020). Five ways to ensure that models serve society: a manifesto. Nature, 582(7813), 482-484. https://doi.org/10.1038/d41586-020-01812-9}

^{UOC R&I} 

^{The UOC's research and innovation (R&I) are helping 21st-century global societies to overcome pressing challenges by studying the interactions between ICT and human activity, with a specific focus on e-learning and e-health. Over 400 researchers and 48 research groups work among the University's seven faculties and three research centres: the Internet Interdisciplinary Institute (IN3), the eLearn Center (eLC) and the eHealth Center (eHC).}

^{The United Nations' 2030 Agenda for Sustainable Development and open knowledge serve as strategic pillars for the UOC's teaching, research and innovation. More information:research.uoc.edu.}