UN Secretary Ban Ki-moon “energized” by students’ “serious discussions” on autonomous weapons systems

UN Secretary General Ban Ki-moon addresses National Model UN conference in the General Assembly Room, 26 March 2015. Photo: NMUN.

In less than two weeks, diplomats from around the world will gather at the United Nations in Geneva to discuss potential global regulations on “lethal autonomous weapons systems” that would be able to select and attack targets without direct human control.

But last week, at the National Model UN conference in New York, attended by some 2,500 undergraduate students from all over the world, a simulation of the UN General Assembly passed three resolutions calling for states to take action to prevent the threat of these “killer robots” to security, human rights and humanitarian law.

Addressing the closing ceremony of the conference, UN Secretary General Ban Ki-moon told students he was “energized by this dynamic gathering” and its “serious discussions” on “cutting-edge issues on the international agenda”, such as “lethal autonomous weapons systems.”

“You are not just leaders of the future – you can start to lead right now,” he told them, “now is the time for your generation to build human solidarity around the world.”

The NMUN NY resolutions defined lethal autonomous robots as “weapons that can select and attack targets independently – without meaningful human input or control”, suggested all countries immediate adopt a national moratorium on such weapons, and urged the negotiation of an international ban through an additional Protocol VI at the Convention on Certain Conventional Weapons (NMUN NY 2015A/GA1-1-1).

Model UN is a simulation of diplomacy, negotiation and decisionmaking by international organizations. Students play the role of diplomats from Member States of the UN and discuss issues at the top of the global policymaking agenda. NMUN NY is one of the biggest undergraduate Model UN conferences in the world.

The students assigned to simulate the General Assembly First Committee – which deals with issues of disarmament and international security – spent several months learning about their countries’ policy positions, the General Assembly and the politics of killer robots. (See for example, their background guide). After debate and drafting in the First Committee, the resolutions were passed by students representing the full plenary body in the actual General Assembly Room at the UN in New York.

The resolutions also called attention to the “work and expertise” of civil society, particularly the Campaign to Stop Killer Robots, Human Rights Watch and the International Committee for Robot Arms Control (ICRAC) (NMUN NY 2015A/GA1-1 -1, GA1-1-2 and GA1-1-3).

In a briefing, Dr. Matthew Bolton, Model UN advisor for Pace University New York City and member of ICRAC, told students at the conference that when new weapons technologies are not adequately addressed by existing regulations, the Marten’s Clause in international law requires states to be guided by “the principle of humanity and the dictates of public conscience.”

“Avoid the temptation to think this simulation is a meaningless game,” said Bolton, “A statement of strong concern from you could be considered an expression of public conscience – a challenge to policymakers in the real world to take action against killer robots.”

ICRAC welcomes the historic decision taken by nations to begin international discussions on how to address the challenges posed by fully autonomous weapons. The agreement marks the beginning of a process that the campaign believes should lead to an international ban on these weapons to ensure there will always be meaningful human control over targeting decisions and the use of violent force.

At 16:48 on Friday, 15 November 2013, at the United Nations in Geneva, states parties to the Convention on Conventional Weapons adopted a report containing a decision to convene on May 13-16, 2014 for their first meeting to discuss questions related to “lethal autonomous weapons systems” also known as fully autonomous weapons or “killer robots.” These weapons are at the beginning of their development, but technology is moving rapidly toward increasing autonomy.

“This is a very significant step forward for the International Committee for Robot Arms Control (ICRAC ),” said Professor Noel Sharkey, Chairman of ICRAC. “We are now on the first rung of the international ladder to fulfill our goal of stopping these morally obnoxious weapons from ever being deployed.”

ICRAC was formed in 2009 to initiate international discussion on autonomous weapons systems. It is made up of experts in robotic technology, artificial intelligence, computer science, international security and arms control, ethics and international law. It is a co-founder of the Campaign to Stop Killer Robots.

The Campaign to Stop Killer Robots believes that robotic weapons systems should not be making life and death decisions on the battlefield. That would be inherently wrong, morally and ethically. Fully autonomous weapons are likely to run afoul of international humanitarian law, and that there are serious technical, proliferation, societal, and other concerns that make a preemptive ban necessary.

“Law follows technology.  With robotic weapons, we have an rare opportunity to regulate a category of dangerous weapons before they are fully realized and the CCW is our best opportunity for regulation,” said Dave Akerson an ICRAC legal expert.

A total of 117 states are party to the Convention on Conventional Weapons, including nations known to be advanced in developing autonomous weapons systems: United States, China, Israel, Russia, South Korea, and United Kingdom. Adopted in 1980, this framework convention contains five protocols, including Protocol I prohibiting non-detectable fragments, Protocol III prohibiting the use of air-dropped incendiary weapons in populated areas, and Protocol IV, which preemptively banned blinding lasers.

“This is a momentous opportunity to get states on the record and behind a ban on fully autonomous offensive weapons,” said Heather Roff, an ICRAC philosopher. “If we can gain enough support, we might succeed in banning a technology before it actually harms innocent civilians.”

The agreement to begin work in the Convention on Conventional Weapons could lead to a future CCW Protocol VI prohibiting fully autonomous weapons.

ICRAC with the Campaign to Stop Killer Robots supports any action to urgently address fully autonomous weapons in any forum. The decision to begin work in the Convention on Conventional Weapons does not prevent work elsewhere, such as the Human Rights Council.

Since the topic was first discussed at the Human Rights Council on 30 May 2013, a total of 44 nations have spoken publicly on fully autonomous weapons since May: Algeria, Argentina, Australia, Austria, Belarus, Belgium, Brazil, Canada, China, Costa Rica, Cuba, Ecuador, Egypt, France, Germany, Ghana, Greece, Holy See, India, Indonesia, Iran, Ireland, Israel, Italy, Japan, Lithuania, Madagascar, Mexico, Morocco, Netherlands, New Zealand, Pakistan, Russia, Sierra Leone, Spain, South Africa, South Korea, Sweden, Switzerland, Turkey, Ukraine, United Kingdom, and United States. All nations that have spoken out have expressed interest and concern at the challenges and dangers posed by fully autonomous weapons.

Together with the Campaign to Stop Killer Robots, ICRAC urges nations to prepare for extensive and intensive work next year, both within the CCW and outside the CCW context.  We urge states to develop national policies, and to respond to the UN Special Rapporteur on Extrajudicial Executions’ call for national moratoria on fully autonomous weapons. We urge states to come back one year from now and agree to a new mandate to begin negotiations. The new process must be underscored by  a sense of urgency.

Peter Asaro, vice-chairman of ICRAC said “The actions of the CCW this week are a hopeful first step towards an international ban on autonomous weapons systems.’

Mathew Bolton delivered a statement on behalf ICRAC at the UN CCW meeting yesterday. As a group of experts we are prepared to help any nations with expert discussions of autonomous weapons systems and to help develop clear definitions for the language to be used in a treaty to ban them. Video footage of the statement, ICRAC’s first ever statement in an official diplomatic forum, is available here.

ICRAC recently coordinated the circulation of a “Scientists Call” to ban fully autonomous weapons systems, signed by more than 270 Computer Scientists, Engineers, Artificial Intelligence experts, Roboticists and professionals from related disciplines in 37 countries, saying: “given the limitations and unknown future risks of autonomous robot weapons technology, we call for a prohibition on their development and deployment. Decisions about the application of violent force must not be delegated to machines.”

International Community Must Act Now to Ensure Combat Will Never Be Outsourced to “Killer Robots”! —

I am speaking on behalf of the International Committee for Robot Arms Control (ICRAC), a founding NGO member of the Campaign to Stop Killer Robots. Since its establishment in 2009, ICRAC has urged the international community to discuss the prohibition of fully autonomous weapons systems – “Killer Robots” – in light of the pressing dangers they pose to global peace and security, in addition to their alarming humanitarian implications for civilians threatened with armed violence.

ICRAC is made up of experts in robotic technology, artificial intelligence, computer science, international security and arms control, ethics and international law. As an indication of our concentration of expertise, over 80% of our members have doctoral or Juris Doctor degrees. As such, ICRAC is available and willing to provide technical expertise to the High Contracting Parties as they engage in further discussions about fully autonomous weapons systems.

ICRAC has coordinated the circulation of a “Scientists Call” to ban fully autonomous weapons systems, signed by more than 270 Computer Scientists, Engineers, Artificial Intelligence experts, Roboticists and professionals from related disciplines in 37 countries, saying: “given the limitations and unknown future risks of autonomous robot weapons technology, we call for a prohibition on their development and deployment. Decisions about the application of violent force must not be delegated to machines.”

ICRAC urges High Contracting Parties to be guided by principles of humanity in its deliberations on existing and emerging weapons technologies –  taking into account considerations of human security, human rights, human dignity, humanitarian law and the public conscience, as well as the justified worries about robotic arms races and proliferation. This means meaningful human deliberation and control over the use of violence must remain the cornerstone of any eventual global policymaking on robotic weapons.

ICRAC welcomes the presentation of the Draft Mandate text enabling the Convention on Certain Conventional Weapons to convene expert meetings in 2014 to look at the challenges posed by fully autonomous weapons systems. ICRAC also welcomes the statements by more than 40 States which have expressed concerns about autonomous weapons systems. We believe the Convention on Certain Conventional Weapons represents a useful forum to begin discussions, though dialogue about autonomous weapons systems need not be confined to this body alone. [END OF ORAL VERSION OF STATEMENT]

In light of our concerns, ICRAC respectfully submits the following recommendations to improve and strengthen the proposed Mandate on autonomous weapons systems before the body:

1)      It is not enough to consider only “lethal” autonomous weapons. We urge the body to broaden the Mandate to include consideration of all autonomous weapons systems, given that:

a) Arming autonomous systems with so-called “sub-lethal” or “less-than-lethal” weapons could still cause unnecessary suffering, and

b) Autonomous weapons systems could be potentially destabilizing even if they are only programmed to attack materiel, particularly if such systems interacted with each other in unexpected ways.

2)      By covering only “emerging technologies”, the Mandate may discourage High Contracting Parties from considering the implications of existing precursor autonomous systems that nonetheless raise unsettling humanitarian questions.

3)      While we welcome an Informal Meeting, a Mandate for a Group of Governmental Experts would more clearly send a message of the seriousness of this matter to the High Contracting Parties.

4)      As experts in this field, we believe the legal and ethical complexities of autonomous weapons systems will be difficult to cover adequately in three days. Thus we suggest amending the Draft Mandate to allow for a five (5) day meeting.

Delivered on behalf of the International Committee for Robot Arms Control, by:

Matthew Bolton, MSc. (LSE), PhD (LSE).
Membership Secretary, International Committee for Robot Arms Control
Advisor, Article 36
Assistant Professor of Political Science, Dyson College of Arts and Sciences, Pace University New York City.

For a PDF version of this statement, click here.

Video footage of the oral statement at the UN in Geneva is available for download here.

Last week the United Nations General Assembly voted overwhelmingly to adopt the Arms Trade Treaty (ATT), which will aim to constrain the flow of conventional weapons to states and organizations that threaten peace and security or engage in gross violations of human rights and humanitarian law.

Several members of the International Committee for Robot Arms Control (ICRAC) – Wim Zwijnenburg of IKV Pax Christi, Thomas Nash and Richard Moyes of Article 36 and Matthew Bolton of Pace University New York City – were engaged in supporting the advocacy work of Control Arms, the global civil society coalition campaigning for a ‘bulletproof’ treaty.

Pushing states to develop text that would cover emerging weapons technologies was a particular emphasis of ICRAC members’ lobbying at the July 2012 and March 2013 Diplomatic Conferences. Many campaigners and diplomats were concerned that the draft treaty did not include specific provisions for ‘unmanned’ weapons, such as aerial drones, or robotic systems that have ‘dual uses.’ A recent report from the Stockholm International Peace Research Institute (SIPRI) raised concerns that the text “looks dangerously likely to be a relic before it ever comes into force.”

Futureproofing

Drawing on technical advice from other ICRAC members, Zwijnenburg and Bolton wrote a policy brief titled “Futureproofing the Draft Arms Trade Treaty” that called on states to make five “critical changes” to the text “in order to cover the emerging class of robotic, ‘unmanned’ and autonomous weapons.” The paper was distributed widely in the conference and online to governments and civil society organizations and was reprinted in Reaching Critical Will’s widely read ATT Monitor newsletter (pp. 3-4). The phrase “futureproofing” caught on and was soon being used widely by Control Arms campaigners, Amnesty International and even the representative of the Holy See.

Not all of the changes suggested in Zwijnenburg and Bolton’s “Futureproofing” paper were made in the final treaty text and it would be disingenuous to overstate ICRAC’s impact. However, by helping to shape and frame the conversation, the policy brief, amplified by Control Arms lobbying, contributed to efforts that changed the treaty text to allow for the future conferences of States Parties to the treaty to review “developments in the field of conventional arms” (Article 17) and adopt amendments by three-quarters vote instead of consensus (Article 20). This means that activists and advocacy organizations will be able to push states to amend the treaty to address developing new weapons technologies. This new text has essentially created a forum in which ICRAC and other stakeholders concerned about emerging weapons technologies can press their case in the future.

What Next?

The next push for campaigners will be to make sure states sign and ratify the ATT, to make it enter into force as quickly as possible. Another important area for advocacy will be to push for a broadening of the categories used by the UN Register of Conventional Weapons. The ATT relies on these categories, which at the moment do not explicitly cover many types of robotic weapons. If civil society can push for states to include unmanned armed systems in this register before the treaty enters into force, the treaty will actually cover a broader scope of weapons.

While the ATT and broadening the UN Register have not been the primary focus of ICRAC’s advocacy, they are establishing important precedents and norms that provide important foundations for the regulation of robotic weapons. Indeed, passing the treaty in a majority vote in the UN General Assembly, instead of consensus, has opened the possibility of developing arms control instruments with high standards, instead of the lowest common denominator.

The ATT is not really a disarmament treaty – it is more of an amalgamation of humanitarian and trade law. Even if it works well, it will only regulate the flows of weapons, not the kind of weapons in circulation. As a result, those who are concerned about the trends toward ‘autonomy’ in robotic weapons, threatening to reduce direct human control over killing, cannot rely on the ATT to prevent this dangerous possibility. This is one of many reasons why ICRAC is part of a growing number of NGOs and faith groups calling for a specific on ban fully autonomous armed robots – “killer robots.”

ICRAC is an international committee of experts in robotics technology, robot ethics, international relations, international security, arms control, international humanitarian law, human rights law, and public campaigns, concerned about the pressing dangers that military robots pose to peace and international security and to civilians in war.