LONDON (28 May 2020) — The computer code behind the scientific modelling of epidemics like COVID-19 should meet independent professional standards to ensure public trust, according to BCS, The Chartered Institute for IT.
The lack of widely accepted software development standards in scientific research has allowed for the 'politicised' undermining of confidence in computational modelling, including in high-profile models informing COVID-19 policy, the Institute argues.
In a new policy paper, BCS calls for professional software development standards to be adopted for research that has a critical impact on society, like health, criminal justice and climate change. The underlying code should also be made open-source.
Dr Bill Mitchell OBE, Director of Policy at BCS, The Chartered Institute for IT said: "The politicisation of the role of computer coding in epidemiology has made it obvious that our understanding and use of science relies as much on the underlying code as on the underlying research.
"We welcome the government's commitment to following science in developing policy responses to the coronavirus pandemic. We support the use of computational modelling in exploring possible outcomes of policy decisions, such as investigating which lockdown measures are likely to have the greatest public health benefits.
"At the same time we consider that - at present - the quality of the software implementations of scientific models appear to rely too much on the individual coding practices of the scientists (who are not computer scientists) who develop them, rather than professional software development practices being publicly evidenced against appropriate standards."
Dr Mitchell cites an example of this issue - a recent article in the journal Nature reports that 'Many machine-learning papers fail to perform an adequate set of experiments', which has led to poor quality research being published. This is also highlighted in a BBC news article that claims 'machine learning [is] causing [a] science crisis', again because some neural networks are being developed that are not following computer engineering best practice. Another recent Nature article has called for scientists to "publicly share both data and code, making it easier for others to attempt to reproduce a paper's findings".
Dr Mitchell continues: "We believe professional software development standards should be followed when implementing computational models for conducting scientific research where that research could be relied on by policy makers and which could have critical consequences for society, such as for example healthcare, criminal justice or climate change. The underlying software code should also be open sourced in line with government guidance."
According to BCS, professionalizing and using best practice software development in scientific research should lead to:
Given the seriousness of this issue and the significant consequences of not using relevant best practice and specialists, BCS will approach experts across the sector to discuss how to professionalise software development practice in scientific research including the Centre for Data Ethics and Innovation, the Alan Turing Institute, the Safety Critical Systems club, the British Insurance Association, Royal Society, the Royal Academy of Engineering, Cabinet Office, NHSX, UKRI, Public Health England.
