Utrecht University: The rise of autonomous defence systems
Ten high-profile speakers gathered on Wednesday 8 June at Utrecht University’s University Hall to discuss what is arguably the most significant and urgent development in warfare today: the rise of autonomous defence systems.
As artificial intelligence and algorithmic technologies are increasingly being integrated into defence systems worldwide, pressing questions concerning the development, deployment, impact and regulation of these new technologies arise. Organised by a group of students and researchers from Utrecht University’s Intimacies of Remote Warfare(IRW) programmein partnership with the Advisory Council for International Affairs (AIV), the goal of the Realities of Algorithmic Defence Symposium was to have different stakeholders engage in an informed, open conversation about these questions, thereby gaining insight into the current state of autonomy in defence systems and the political, ethical, and regulatory challenges which accompany the integration of AI and algorithmic technologies by advanced militaries worldwide.
As co-author of the essential AIV and CAVV report on autonomous weapon systems, Prof. Dr. Cedric Ryngaert, Professor of Public International Law, opened the symposium by outlining the content and key messages of the report. Hereafter, the symposium proceeded with three panels, each focusing on a different dimension of the larger topic: development, analysis and regulation.
The three speakers of the first panel could not have been more qualified to represent the field developers and practitioners: Lieutenant-Colonel Martijn Hädicke, commander of the Robotics and Autonomous Systems unit of the Royal Netherlands Army, Maurits Korthals Altes, founder of robotics startup and MoD contractor Avalor AI, and Dr. Nanda van der Stap, senior scientist innovator and team lead at TNO. Moderated sharply and in an engaging fashion by Director of Research at The Hague Centre for Strategic Studies Dr. Tim Sweijs, the panel provided a comprehensive insider’s perspective on how different parties collaborate in developing autonomous defence systems and why they perceive these technologies to be important. An interesting insight, that was referred back to various times throughout the event, was provided by Korthals Altes, who stated that the algorithms created by Avalor AI are exclusively trained in simulated settings, as there is very little data available that sufficiently grasps the complexity and unpredictability inherent to real-life warzones.
Critical analysis
Following up on the picture portrayed by the first panel, the second panel discussed the need for critical analysis of autonomous weapon systems. With Maaike Verbruggen, Doctoral Researcher at the Vrije Universiteit Brussel, Dr. Marijn Hoijtink, Assistant Professor in International Relations and International Security at the Vrije Universiteit Amsterdam, and Jennifer Gibson, US lawyer and lead of NGO Reprieve’s extrajudicial executions team, the expertise and academic authority present in this panel was impressive. Moderated by IRW’s own Dr. Lauren Gould, the panel members expressed critical concern over the fact that Western armies are conducting signature strikes with drones, based on metadata and algorithms. Also deemed alarming was the fact that they do not investigate strikes nor disclose sufficient information on who was harmed. Without collecting post-strike data, armies know little about the identities of those who they killed, there is no feedback loop to adequately train algorithms and machine learning, nor can civilians seek accountability for the loss of their loved ones. This raises grave concerns that the current AI arms race amongst advanced militaries will only exasperate the wrongful targeting of innocent civilians.
Unilateral agreements
In the third panel, the political, legal, and ethical dimensions of regulating autonomy in weapon systemswere explored. Moderated by Dr. Marijn Hoijtink, the panel consisted of three speakers: the aforementioned Prof. Dr. Cedric Ryngaert, Jessica Dorsey, Assistant Professor of the International and European law department at Utrecht University, and Frank Slijper, who researches the proliferation of autonomous weapons and the international arms trade at NGO PAX. Building on the realities brought to light during the first two panels, Jessica Dorsey held a convincing plea to “first lay down the guidelines, and then develop and deploy autonomous weapons systems”. However, the first insight from the panel was that the options to realize this are limited. Especially challenging is that the current domestic and international legal frameworks do not pose great opportunities for successful regulation of autonomous weapon systems and the formation of a new multilateral treaty on autonomous weapon systems is unlikely. Yet, there are also possibilities for international regulatory progress: by coming to unilateral agreements on regulation, like-minded countries can pressure other states to adopt a high standard of practices concerning the use and development of autonomous weapon systems.
Highly relevant because of its topic, insightful in its content and unique in the quality of the speakers assembled, the RAD symposium served its purpose: having stakeholders sit together to discuss the current state of autonomy in defence systems in all its most important dimensions, exploring what is here and now, establishing the realities of algorithmic defence.