Technical University of Munich Working Group Establishes Guidelines to Ensure Trust in AI-Driven Research
Science thrives on reproducibility, transparency, and accountability, and trust in research stems in particular from the fact that results are valid regardless of the institution where they were produced. Furthermore, the underlying data of a study must be published, and researchers must take responsibility for their publications.
But what if AI is involved in the research? Experts have long used AI tools to design new molecules, evaluate complex data, and even generate research questions or prove a mathematical conjecture. AI is changing the face of research, and experts are debating whether the results can still be trusted.
Recommendations published in the scientific journal PNAS
Five principles should continue to ensure human responsibility in research, according to an interdisciplinary working group with members from politics, business, and academia published in the latest issue of the journal Proceedings of the National Academies of Sciences (PNAS). Urs Gasser, Professor for Public Policy, Governance and Innovative Technology at TUM, was one of the experts.
The recommendations in brief:
- Researchers should disclose the tools and algorithms they used and clearly identify the contributions of machines and humans.
- Researchers remain responsible for the accuracy of the data and the conclusions they draw from it, even if they have used AI analysis tools.
- AI-generated data must be labeled so that it cannot be confused with real-world data and observations.
- Experts must ensure that their findings are scientifically sound and do no harm. For example, the risk of the AI being “biased” by the training data used must be kept to a minimum.
- Finally, researchers, together with policymakers, civil society and business, should monitor the impact of AI and adapt methods and rules as necessary.
“Signal effect for researchers”
“Previous AI principles were primarily concerned with the development of AI. The principles that have now been developed focus on scientific applications and come at the right time. They have a signal effect for researchers across disciplines and sectors,” explains Urs Gasser.
The working group suggests that a new strategy council – based at the US National Academy of Sciences, Engineering and Medicine – should advise the scientific community.
“I hope that science academies in other countries – especially here in Europe – will take this up to further intensify the discussion on the responsible use of AI in research,” says Urs Gasser.