Karlsruher Institute for Technology: Computer scientists at KIT and HITS develop open source tools that automatically check development standards for scientific software – 48 tools are evaluated
Digital tools are essential in almost all scientific disciplines. Reliable, carefully developed software is particularly important where large amounts of research data accumulate and have to be processed quickly in order to analyze and interpret them correctly – for example in biology, physics or, more recently, in epidemiology. However, the development and maintenance of scientific software is often characterized by insecure financing and time pressure, which can lead to quality defects. In order to evaluate the software quality automatically, computer scientists from the Karlsruhe Institute of Technology (KIT) and the Heidelberg Institute for Theoretical Studies (HITS) designed the SoftWipe tool. They have published their results in the Nature Scientific Reports Journal .
“Compliance with coding standards is far too seldom taken into account in scientific software, although this can even lead to false scientific results,” says Professor Alexandros Stamatakis, who works at both the HITS and the Institute for Theoretical Computer Science (ITI) at KIT. The open source tool SoftWipe offers a fast, reliable and cost-effective solution by automatically evaluating software development standards. In addition to the tool itself, the computer scientists compiled a ranking of 48 scientific software from various research areas and assessed whether the coding standards were adhered to.
“SoftWipe can also be used in the review process of scientific software and support the selection process,” adds Adrian Zapletal. The master’s student and his fellow student Dimitri Höhler made a significant contribution to the development of SoftWipe. When selecting the evaluation criteria, they were based on existing standards that are used in safety-critical areas, such as at NASA or at CERN.
“In our investigations we found enormous differences in software quality,” says co-author Professor Carsten Sinz from the ITI. Many programs, such as covid-sim, which is used in Great Britain for the mathematical modeling of the infectious disease COVID-19, had a very low quality score and thus did poorly in the ranking. The scientists recommend that programs such as SoftWipe should be used as standard in the selection and assessment process for software in the scientific field.
This is how SoftWipe works
SoftWipe is a pipeline written in the Python3 programming language that uses several mostly freely available static and dynamic code analyzers to evaluate the code quality of software written in C / C ++. SoftWipe compiles the software and executes it in order to detect programming errors during execution. Based on the output of the tools used for code analysis, SoftWipe calculates a quality score between 0 (poor) and 10 (very good) and creates an overall rating from this.