The results were totally clear cut with an overwhelming 76% of engineers voting that there should be an unequivocal ban on developing ‘killer robots’.

The next-largest group, 14 per cent, had a similar view. They voted that attack logisitics could be autonomous as long as target selection remained under human control.

Ron Arkin from GIT was specifically mentioned in the poll. He has long opposed our call for a ban and has said that we should have a moratorium on autonomous weapons while control systems are being perfected. The engineers were not impressed – only 3% voted in his favour.

The other two options: autonomy for all systems other than weapons, or for armed forces being unwilling to adopt unmanned armed systems, failed to get more than 5 per cent of the vote. We would not mind autonomy for all systems other than weapons but feel happier that we got a resounding YES from The Engineer for our campaign. THANK YOU!

Read the full story in The Engineer

Several members of the International Committee for Robot Arms Control (ICRAC) – Wim Zwijnenburg of IKV Pax Christi, Thomas Nash and Richard Moyes of Article 36 and Matthew Bolton of Pace University New York City – were engaged in supporting the advocacy work of Control Arms, the global civil society coalition campaigning for a ‘bulletproof’ treaty.

Pushing states to develop text that would cover emerging weapons technologies was a particular emphasis of ICRAC members’ lobbying at the July 2012 and March 2013 Diplomatic Conferences. Many campaigners and diplomats were concerned that the draft treaty did not include specific provisions for ‘unmanned’ weapons, such as aerial drones, or robotic systems that have ‘dual uses.’ A recent report from the Stockholm International Peace Research Institute (SIPRI) raised concerns that the text “looks dangerously likely to be a relic before it ever comes into force.”

Futureproofing

Drawing on technical advice from other ICRAC members, Zwijnenburg and Bolton wrote a policy brief titled “Futureproofing the Draft Arms Trade Treaty” that called on states to make five “critical changes” to the text “in order to cover the emerging class of robotic, ‘unmanned’ and autonomous weapons.” The paper was distributed widely in the conference and online to governments and civil society organizations and was reprinted in Reaching Critical Will’s widely read ATT Monitor newsletter (pp. 3-4). The phrase “futureproofing” caught on and was soon being used widely by Control Arms campaigners, Amnesty International and even the representative of the Holy See.

Not all of the changes suggested in Zwijnenburg and Bolton’s “Futureproofing” paper were made in the final treaty text and it would be disingenuous to overstate ICRAC’s impact. However, by helping to shape and frame the conversation, the policy brief, amplified by Control Arms lobbying, contributed to efforts that changed the treaty text to allow for the future conferences of States Parties to the treaty to review “developments in the field of conventional arms” (Article 17) and adopt amendments by three-quarters vote instead of consensus (Article 20). This means that activists and advocacy organizations will be able to push states to amend the treaty to address developing new weapons technologies. This new text has essentially created a forum in which ICRAC and other stakeholders concerned about emerging weapons technologies can press their case in the future.

What Next?

The next push for campaigners will be to make sure states sign and ratify the ATT, to make it enter into force as quickly as possible. Another important area for advocacy will be to push for a broadening of the categories used by the UN Register of Conventional Weapons. The ATT relies on these categories, which at the moment do not explicitly cover many types of robotic weapons. If civil society can push for states to include unmanned armed systems in this register before the treaty enters into force, the treaty will actually cover a broader scope of weapons.

While the ATT and broadening the UN Register have not been the primary focus of ICRAC’s advocacy, they are establishing important precedents and norms that provide important foundations for the regulation of robotic weapons. Indeed, passing the treaty in a majority vote in the UN General Assembly, instead of consensus, has opened the possibility of developing arms control instruments with high standards, instead of the lowest common denominator.

The ATT is not really a disarmament treaty – it is more of an amalgamation of humanitarian and trade law. Even if it works well, it will only regulate the flows of weapons, not the kind of weapons in circulation. As a result, those who are concerned about the trends toward ‘autonomy’ in robotic weapons, threatening to reduce direct human control over killing, cannot rely on the ATT to prevent this dangerous possibility. This is one of many reasons why ICRAC is part of a growing number of NGOs and faith groups calling for a specific on ban fully autonomous armed robots – “killer robots.”

ICRAC is an international committee of experts in robotics technology, robot ethics, international relations, international security, arms control, international humanitarian law, human rights law, and public campaigns, concerned about the pressing dangers that military robots pose to peace and international security and to civilians in war.

For example, they must not be equipped with biological or chemical weapons. In Europe most land and air vehicles are covered by the definitions of the Treaty on Conventional Armed Forces (CFE Treaty), thus they are limited by numbers and subject to verification. If armed uninhabited vehicles cannot be prohibited outright, then limitations similar to the CFE Treaty are needed in other regions of the world. To avoid dangers for international humanitarian law and military stability, autonomous attack, that is attacks without a human decision in each single case, should be prohibited. Additional prohibitions are needed, among others, for small and very small armed vehicles.

Jürgen Altmann, Arms control for armed uninhabited vehicles: an ethical issue, Ethics and Information Technology, 2013, DOI 10.1007/s10676-013-9314-5. Read the full article  (open access, 17 pages) here.

Latiff and McCloskey point out the military benefits of the autonomous machines and then comes the but…

The problem is that robotic weapons eventually will make kill decisions on the battlefield with no more than a veneer of human control. Full lethal autonomy is no mere next step in military strategy: It will be the crossing of a moral Rubicon. Ceding godlike powers to robots reduces human beings to things with no more intrinsic value than any object.

When robots rule warfare, utterly without empathy or compassion, humans retain less intrinsic worth than a toaster—which at least can be used for spare parts. In civilized societies, even our enemies possess inherent worth and are considered persons, a recognition that forms the basis of the Geneva Conventions and rules of military engagement.

Lethal autonomy also has grave implications for democratic society. The rule of law and human rights depend on an institutional and cultural cherishing of every individual regardless of utilitarian benefit. The 20th century became a graveyard for nihilistic ideologies that treated citizens as human fuel and fodder.

They speak very frankly about the recent US department of defense directive:

The kill decision is still subject to many layers of human command, and the U.S. Defense Department recently issued a directive stating that emerging autonomous weapons “shall be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force.”

Yet this seems more like wishful thinking than realistic doctrine. Military budget cuts are making robotic autonomy almost fiscally inevitable.

There is a solid criticism of Michael N. Schmitt’s (Chairman of the US Naval War College International Law Department) view that war machines can protect civilian and property as well as humans. “This assurance aside, it is far from clear whether robots can be programmed to distinguish between large children and small adults, and in general between combatants and civilians, especially in urban conflicts. Surely death by algorithm is the ultimate indignity.”

The conclusion of the article is strongly in line with the Campaign to Stop Killer Robots:

Time is running out for military decision makers, politicians and the public to set parameters for research and deployment that could form the basis for national policy and international treaties. The alternative is to blindly accept as inevitable whatever technology offers. Let’s not be robotic in our acquiescence.

Read the full article – With drone warfare, America approaches the Robot-rubicon

For the ban, he notes that

The arguments against developing fully autonomous weapons, as they are called, range from moral (“they are evil”) to technical (“they will never be that smart”) to visceral (“they are creepy”).

“This is something people seem to feel at a very gut level is wrong,” says Stephen Goose, director of the arms division of Human Rights Watch, which has assumed a leading role in challenging the dehumanizing of warfare. “The ugh factor comes through really strong.”

He then discusses the three International Humanitarian issues with autonomous robot weapons (i) inability to conform to the principle of distinction; (ii) inability to conform to the principle of Proportionality and (iii) difficulties with accountability with mishaps or war crimes.

He brings out the usual suspect, Ron Arkin, to argue against a ban. Arkin still believes that robots could do better than human because they don’t have emotional responses. Others argue that that is one of the main problems. The funniest comment to Keller’s article was a response to Ron Arkin:

Professor Arkin argues that automation can also make war more humane.” This guy has obviously been a civilian all his life. Only a civilian would believe there is a humane way to kill another human being. Does he get out of the house on a regular basis?

But Arkin’s position in other respects does not now seem that removed from those calling for a ban. “He advocates a moratorium on deployment and a full-blown discussion of ways to keep humans in charge.” The human’s in charge is a subtle change in Arkin’s position that is greatly appreciated. It moves us some way toward the discussions that should be had.

However, without a ban on the development and research on these weapons systems, they are going to end up in the US arsenal. Other countries have not said that they will have a moratorium and so we can expect and arms race that the US will not be able to resist.

In fact in terms of a moratorium, Keller appears to have made an error of interpretation with regards to the recent Department of Defence directive (November 21 2012) ” Last November the Defense Department issued what amounts to a 10-year moratorium on developing them while it discusses the ethical implications and possible safeguards.”

ICRAC member Mark Gubrud picks up on this error in a comment after Keller’s piece:

The DoD Directive (3000.09) does not impose any moratorium. It says that the United States will develop and use autonomous weapons.

Although it draws a line at AW that kill humans autonomously, it does not forbid crossing the line; rather, it sets forth the procedure for doing so. Four sub-cabinet level signatures are required. Other than that, the rules for AW that kill humans are essentially the same as for AW that target materiel, which the Directive approves already.

The directive also approves for immediate development and use “semi-autonomous weapons” which may automatically acquire, track, identify and prioritize potential targets, cue a human operator to their presence, and upon approval, engage them, automatically determining the timing of when to fire.

So, a semi-autonomous weapon system might detect a group of persons, highlight their dim outlines on a screen, and say to the operator “target group identified.” The operator says “engage” and the machine kills them.

Such a system already has every capability needed for full lethal autonomy. It has only been programmed to request approval. One trivial software modification will fix that, if the system doesn’t already have a switch to throw it into full autonomous mode.

DoDD 3000.09 approves such systems for immediate development, acquistion and use.

There is no moratorium; it is a full-speed charge into the unknown.

Nonetheless, Keller is clearly on the right side of the issues and shows a clear understanding: ” It’s a squishy directive, likely to be cast aside in a minute if we learn that China has sold autonomous weapons to Iran”

Although Keller is not optimistic about the chance of us getting a ban on killer robots, he supports it and ICRAC appreciates him for that:

I don’t hold out a lot of hope for an enforceable ban on death-dealing robots, but I’d love to be proved wrong. If war is made to seem impersonal and safe, about as morally consequential as a video game, I worry that autonomous weapons deplete our humanity. As unsettling as the idea of robots’ becoming more like humans is the prospect that, in the process, we become more like robots.

It is well worth reading Bill Keller’s full story and the comments that come afterwards – Smart Robots.

“Obama administration officials told CNN that the United States is concerned with Iranian intentions in incidents like this and whether such episodes could unintentionally trigger hostilities.”

This is just the kind of confrontation that ICRAC members have warned about happening when autonomous drones are deployed in the near future – drones that once launched will complete their mission without further intervention.

Without a human in the control loop to accept or issue warnings misunderstanding between software systems could lead to inadvertent armed confrontations that could trigger unintended large scale hostilities.

Russian Today: Medal of Droner

Now CNN has reported that Panetta’s successor, Chuck Hagel who has two purple hearts, has set up a review that will most likely lower the medal’s ranking. Read the CNN story - Hagel ordering review of new drone medal ranking

“Much of the conversation about the impact of these strikes has rightly focused on the moral and legal costs of these civilian casualties, but it is a mistake to judge the impact of the U.S. drone program only by the number of sorties or kills. When this is the sole basis for evaluation, it is easy to argue that there is nothing particularly unique about this form of warfare – that these people would have been targeted and killed by U.S. Special Forces or manned aircraft had the drone program not been in place.

But this type of analysis misses a defining characteristic of the drone program that makes it qualitatively different from the less sophisticated weaponry that it is replacing: Ubiquitous drone use blurs the line between citizen and militants.

The psychological impact of drone surveillance, when combined with the civilian casualties we already know occur during strikes, leads to significant negative strategic costs that need to be incorporated into our assessment of the drone program.”

The author, Taylor Owen, extensively uses the report Living Under Drones, by our friends at Stanford and NYU law and commissioned by Reprieve. But he adds new insights about the psychological trauma in Gaza – anticipatory anxiety keeps people from attending social gatherings such as weddings and funerals. This message cannot be repeated enough

Owen talks about the New York Times Journalist, David Rohde, who was kidnapped by the Taliban:

“David Rohde described both the fear the drones inspired among his captors, as well as among ordinary civilians: “The drones were terrifying. From the ground, it is impossible to determine who or what they are tracking as they circle overhead. The buzz of a distant propeller is a constant reminder of imminent death.”

Read the full article – Drones don’t just kill. Their psychological effects are creating enemies

Say No To Killer Robots

“This is a call to engineers to stand up and demand the prohibition of autonomous lethal targeting by robots. I ask this of engineers because you are the ones who know just how limited machines can be when it comes to making judgments; judgments that only humans should make; judgments about who to kill and when to kill them.”

The aim was to raise awareness of the issues within the engineering community that will be responsible for the development of lethal robots. There are many people with integrity in this community who care deeply about issues involving the killing of innocent civilians. But they might not aware that the artifacts they are working on could be used as autonomous weapons.

“The bottom line is that weapon systems should not be allowed to make decisions to select human targets and engage them with lethal force. We need to act now to stop the kill function from being automated. We have already prohibited chemical weapons, biological weapons, blinding lasers, cluster munitions and antipersonnel landmines.

We now need a new international treaty to pre-emptively ban fully autonomous weapons.

I call on you to sign our call for a ban at http://icrac.net/ before too many countries develop the technology and we venture down a path from which there is no return.”

ICRAC also calls for the same commitment from Computer Scientists, roboticists and professionals from related computing and engineering disciplines.

Read the full story – Say no to killer robots