On April 15, ICRAC’s Juergen Altmann delivered the following statement to the informal “Meeting of Experts“, gathered to discuss questions related to “lethal autonomous weapons systems” from April 11 to April 15 at the United Nations in Geneva, Switzerland.

Thank you for allowing the International Committee for Robot Arms Control to provide a second statement about the problems with AWS and regional security. We are speaking to further John Borrie’s notion of hidden interaction and crisis.

Yesterday ICRAC explained how the interaction between two systems of autonomous weapon systems with secret combat algoritms would produce unpredictable and unanticipated consequenes. This is cautious scientific wording, but there is more to say.

Imagine a severe crisis, with the swarms of adversaries operating in close proximity of each other. A coordinated attack of one could wipe out the other within missile flight time – that is seconds. The control software would have to react fast to repel the attack.

The problem is that an erroneous “counter” attack could be triggered by a sun glint in visual data misinterpreted as a rocket flame, sudden, unforeseen moves of the enemy swarm, or a simple software bug.  That could then lead to a counter-attack of the other side, with fast escalation from crisis to war.

If war is seen as unavoidable, then there would be strong pressures to attack the other swarm first. Such pre-emption and the fear of it would provide another mechanism of fast and uncontrolled escalation.

We cannot determine the mathematical probabilities of such scenarios or simulate them on a computer. But they are certainly possible and plausible. Considerations of this sort were a strong motive for the arms control treaties limiting nuclear weapons.

Crisis instability as described is not limited to the interactions between big military powers – they can as well occur between regional powers. The best way to avoid such destabilisation would be to preventively prohibit autonomous weapon systems.

We ask those states seeing military advantages in AWS to ponder the mid- and long-term threats from autonomous escalation.