POLICIES ON THE EMPLOYMENT OF LETHAL AUTONOMOUS WEAPON ...€¦ · IoT Internet of Things ISR Intelligence, Surveillance, and Reconnaissance LARs Lethal autonomous robotics LAWS Lethal

POLICIES ON THE EMPLOYMENT OF LETHAL AUTONOMOUS WEAPON SYSTEMS

IN FUTURE CONFLICTS

By

Ted W. Schroeder

ii

© COPYRIGHT

by

Ted Schroeder

December 2016

ALL RIGHTS RESERVED

iii

THE IMPLICATIONS OF LETHAL AUTONOMOUS WEAPON SYSTEMS ON FOREIGN

POLICY

BY

Ted Schroeder

ABSTRACT

The conversation on Lethal Autonomous Weapon Systems (LAWS) centers on the ethics

of allowing a computer to decide to kill (or not to kill) a human-being. Much of the current

discourse on the topic of autonomous weapons comes from a concern over the ethical

implications. Over the coming fifteen years, the technology industry will achieve many

milestones that will significantly alter the argument about the use of LAWS. There are currently

efforts to institute laws and regulations that will inhibit or remove the use of LAWS. This

research will clarify what will be technically possible in the future and take a holistic look at the

topic. This study will explore the current technological abilities of Artificial Intelligence (AI)

and its impacts on civil society. It will further look at AI and its impact on lethal weapons. Ad-

ditionally, the study will explore the acceptance of AI in civil society verse the acceptance of AI

in conflict. Such exploration is important as the newer technology may change the conversation

about the ethics of employing robotics. This conversational change may encourage or even

compel policymakers to use LAWS in future conflicts.

iv

Acknowledgements

I am very grateful for the time and help that Paul Gelpi Ph.D. gave me throughout the

process. His patience and knowledge was invaluable. It has been an honor to work with him.

Most of all I am thankful to my wife Sarah. Her constant encouragement, help, and lov-

ing guidance was everything to me throughout this degree. Words can never express the debt

that I owe her.

v

List of Abbreviations

AAAI Association for the Advancement of Artificial Intelligence

AACUS Autonomous Aerial Cargo Utility System

ACTUV ASW Continuous Trail Unmanned Vessel

AI Artificial Intelligence

AGI Artificial General Intelligence

ALIAS Aircrew Labor In-Cockpit Automation System

AMAS Autonomous Mobility Appliqué System

ANI Artificial Narrow Intelligence

ASI Artificial Super Intelligence

ASIMO Advance Step Innovative Mobility

ASW Anti-Submarine Warfare

A-UGV Automated Unmanned Ground Vehicle

CALO Cognitive Assistant that Learns and Organizes

CEV Coherent Extrapolated Volition

CCW Convention on Certain Conventional Weapons

CIWS Close-In Weapon System

CNAS Center for a New American Security

DARPA Defense Advanced Research Projects Agency

DOD Department of Defense

EOD Explosive Ordinance Disposal

HRW Human Rights Watch

ICRC International Committee of the Red Cross

vi

IEEE Institute of Electrical and Electronics Engineers

IHL International Humanitarian Law

IHRC International Human Rights Clinic

ICBM Intercontinental Ballistic Missile

IoT Internet of Things

ISR Intelligence, Surveillance, and Reconnaissance

LARs Lethal autonomous robotics

LAWS Lethal Autonomous Weapons Systems

LIDA Learning Intelligent Distribution Agent

LOAR Law of Accelerating Returns

MUM-T Manned Un-Manned Teaming

MIRI Machine Intelligence Research Institute

NDU National Defense University

NGO Non-Governmental Organization

NLP Natural Language Processing

PLA People’s Liberation Army

RAS Robotic and Autonomous Systems

SIRI Speech Interpretation and Recognition Interface

SyNAPSE Systems of Neuromorphic Adaptive Plastic Scalable Electronics

UCAS Unmanned Combat Air System

UGV Unmanned Ground Vehicles

UN United Nation

UUV Unmanned Undersea Vehicles

vii

Table of Contents

Abstract ………………..………………………………………………………………………iii

Acknowledgements ………………………………………………………..………………….iv

List of Abbreviations……………….…………………………………………………………..v

Table of Contents………………………………………………………………………………vii

Chapter 1: Introduction …………………………………………………………….………….1

Statement of the Problem and Research Question……………………………………..1

Justification ……………………………………………………………………………2

Contribution……………………………………………………………………………2

Overview……………………………………………………………………………….3

Statement of the Probable Value and Importance of the Study ………….……………5

Limitations of the Study………………………………………………….…………….5

Methodology …………………………………………………………………………..6

Framework …………………………………………………..………….……………..6

Literature Review ……………………………………………………….…………….7

Summary……………………………………………………………………………….13

Chapter 2: Understanding Artificial Intelligence……………………………………………...14

Definitions……………………………………………………………………………..14

Computer Visual Recognition…………………………..……………………………..14

Speech Recognition……………………………………………………………………16

Cognitive Computer Processing……………………………………………………….18

Artificial Intelligence Theories or How I Learned to Stop Worrying and Love ASI….23

Summary ………………………………………………………..……….…………….29

Chapter 3: Acceptance of AI Improving Life in the Western World…………………………..30

Applications of AI in Robotics………………………………………………..….…….30

Applications of AI in Other Forms……………………………………………………..32

Summary ……………………………………………………………………………….34

Chapter 4: Lethal Autonomous Weapon Systems in Practice………………………………….35

Currently Employed LAWS……………………………………………………………35

Autonomous Systems Currently Being Developed…………………………………….37

Summary ……………………………………………………………………………….41

Chapter 5: Debate…………………………………………………………………………….....42

Legal & Ethical Aspect………………………………………………………………….42

Political Aspect…………………………………………………………………..……...48

Military Aspect……………………………………………………………………..…...51

Technological Limitations…………………………………………………………..…..55

Differences in Costs Between Humans and Robots.........................................................55

viii

Summary ……………………………………………………………………………......56

Chapter 6 Conclusion……………………………………………………………………………57

Potential Influences of LAWS…………………………………………………………..57

Summary ………………………………………………………………………………..58

Suggested Follow-on Research ………………………………………………................60

Final Thoughts…………………………………………………………………………...61

References…………………………………………………………………………………….....63

1

CHAPTER 1

INTRODUCTION

Statement of Problem and Research Question

Currently, the conversation on Lethal Autonomous Weapons Systems (LAWS) centers

on the ethics of allowing a computer to decide to kill (or not to kill) a human-being. Over the

coming fifteen years, the defense and technology industries will achieve many milestones that

will significantly alter the argument about the use of LAWS. Computer processors will continue

to increase in speed until they are not only on par with the human brain but eventually "smarter"

than 1,000 human brains. Recognition software will move past its current hurdles of recognizing

the difference between an apple and a tomato to distinguishing between happy and content.1

When these innovations become a reality, the world will have machines that are considered

smarter and have better visual recognition than humans. Artificial Super Intelligence (ASI)

powered machines will have an impact on every part of the human experience from medicine to

driving cars.2 Concurrently, many in the public, lobbyists, and special interest groups will

advocate having machines make more decisions in war, thereby removing the human element.

People are emotional whereas autonomous weapons, powered by computers, are programmed to

be just. Political leaders will have to make a new series of decisions that may remove humans

from making the day to day judgments about who is killed during a conflict.

1 Singer, P. W. (2009). Wired for War: The Robotics Revolution and Conflict in the 21st Centu-

ry. (Kindle Locations 1551).

Dwoskin, Elizabeth., Evelyn M. Rusli. (2015).The Technology that Unmasks Your Hidden Emo-

tions Using Psychology and Data Mining to Discern Emotions as People Shop, Watch Ads;

Breeding Privacy Concerns. Tech. The Wall Street Journal. 2 IBM Watson Health. (2016). IBM Watson Health Website.

Google Self-Driving Car. (2016). Google Self-Driving Car Website.

2

The proposed research will take into account the future innovations in technology. It will

also discuss how the aggregate of these developments will effect the likelihood of the use of

LAWS in future warfare. These considerations will be determined by asking the following

question: Will the development of autonomous weapons systems by the military affect political

leaders' willingness of using them in an international conflict?

Justification

As LAWS becomes a technological reality, policymakers will have more opportunities

to deploy a military force with limited to no risk to personnel. The use of drones will continue to

become an integral part of militaries throughout the world. Unmanned systems are currently

being tested for use on waterways and land. Policymakers are leery of committing military

forces if it there is a possiblity of losing servicemembers. As the cost in human capital decreases

with the use of autonomous weapons, policymakers will face new challenges and therefore have

to make different calculations. In today’s environment there is pressure to not allow a robot to

decide whether or not to kill a human. There is also pressure to minimize the risk to life. These

pressures will not abate. The LAWS technology will continue to mature and change dynamics of

the argument. It is important to fully explore the possibilities on both sides of the issue. The

global community will observe how the upcoming changes in the character of warfare will

impact foreign policy and the inclination of political decision makers to use LAWS in conflict.

Contribution

Much of the current discourse on the topic of autonomous weapons comes from a

concern over the moral implications. There are currently efforts to institute laws and regulations

3

that will inhibit or remove the use of LAWS. For example, in April of 2016, a Convention on

Certain Conventional Weapons (CCW) was held in Geneva to collate a list of concerns from

experts about LAWS.3 The convention included speakers from Campaign to Stop Killer Robots

and Human Rights Watch (HRW) which are actively campaigning to ban LAWS.4

This research will clarify what will be technically possible in the future and take a

holistic look at the topic. This study will explore the current technological abilities of Artificial

Intelligence (AI) and its impacts on civil society. It will further look at AI and its impact on

lethal weapons. Additionally, the study will explore the acceptance of AI in civil society verse

the acceptance of AI in conflict. Such exploration is important as the newer technology may

change the conversation about the ethics of using robotics. This conversational change can

encourage or even force the hand of policymakers to use LAWS in future conflicts.

Overview

In the rest of the first chapter, the study will provide a statement of the probable value

and importance of the study. The limitations of the research will clarify the boundaries that were

adhered to throughout the research. Methodology and framework are provided to bring to light

how the research was conducted. The first chapter ends with the literature review, which cap-

tures major works on the topic of LAWS.

In the second chapter, the research will establish an accurate understanding of Artificial

Intelligence. It will include major developments in the field of AI and current research. It will

further examine the published works of distinguished experts in the field.

3 United Nations Office at Geneva. (2016). 2016 Meeting of Experts on LAWS. United Nations.

4 Campaign to Stop Killer Robots. (2016). Campaign to Stop Killer Robots Home Page. Website.

4

The third chapter will provide examples of accepted, life-improving AI technology in the

Western world. It will include the impact that ASI has had in Google and its search engine,

Amazon’s Prime Air, and IBM’s Watson’s medical applications.

In the fourth chapter, the study will examine how militaries around the world are

developing unmanned systems and their potential application in conflicts. A clear discussion on

what unmanned systems can do today will give greater context on what autonomous weapons

will provide for the military during usage in combat. The U.S. has been at the forefront of

development, but many countries throughout the world have their own programs. This paper

will explain what the near-term and long-term future can physically provide the military.

Chapter five will examine the debate over the ethics of developing and using ASI and

LAWS. There are currently two camps on the ASI debate. In one camp the world is a better

place where computers aid humankind in achieving peace; maintaining longer, healthier lives;

and innovating solutions to any problem on earth. The other camp sees ASI as a monster that, at

best, will tolerate the existence of humans, being indifferent to whether they live or die. At

worst, ASI decides that it must eliminate all humankind to save resources for itself.

Chapter five will also look at the current laws and policies that limit autonomous

weapons. It will continue by exploring how laws are codified now and how they may be

changed or adapted in the future. This will lead to ethical questions of whether or not AI or ASI

should be trusted to control a weapon system. There are many different organizations which

have an opinion on this subject. The Non-Governmental Organization (NGO), Human Rights

Watch, is lobbying for a codified law that inhibits the development and use of autonomous

weapons. This is largely based on the lack of trust in modern computers. Politicians, the public,

5

institutions, and the military all have personal incentives that sway ethical views on the

employment of autonomous weapons.

In the final chapter, the research will summarize the arguments for and against LAWS.

This chapter will then make recommendations for further research. The recommendations will

be broken down into the following categories: humanitarian focused NGOs involvement,

political ramifications of employing LAWS, and the benefits and limitations to militaries’

employment of LAWS. This study will end with final thoughts from the author.

Statement of the Probable Value and Importance of the Study

This study will provide a holistic view on the civil discourse about ASI empowered

LAWS in conflicts. Corporations are looking to ASI to improve every aspect of human life. Au-

tonomous robotics are fulling jobs in every industry from hotels to elderly care.5 As ASI em-

powered robotics become a part of everyday life, it is likely they will have an impact on the mili-

tary. As the military industry is able to produce LAWS, governments will look to LAWS to in-

crease their security. As these developments come, it is important to have a complete under-

standing of the capabilities and limitations of LAWS.

Limitations of the Study

This research is limited to the study of physical, unmanned systems that travel through

open spaces. This study will not discuss the ramifications of nanotechnology on foreign policy.

Although similar to other unmanned systems, nanobots require a separate civil discourse not well

5 Neild, Barry. (2016). Why your next hotel will be staffed by robots. CNN.

RT Staff. (2015). Japan to Create More User-Friendly Elderly Care Robots The Japanese gov-

ernment said some of the elderly care robots built have been too large and too expensive.

6

suited to this discussion. Additionally, this study will examine current laws that apply to auton-

omous weapons but will not discuss them in detail. The proper verbiage of the laws that would

dictate the perimeters of LAWS will require a study all their own. Further, this study will avoid

human augmented intelligence. Although it is very close to the discussion on LAWS, it broad-

ens the scope of this research too much to ensure clear fidelity on the discussion. Finally, this

study will not address cyber technology directly. Cyber warfare and the political policies around

them are too broad for this work. Cyber technology will be referenced regularly to give a greater

perspective on how emerging technologies are being viewed on a larger scale.

Methodology

This study will use a discourse analysis to examine the content on providing lethality to

autonomous weapons and political leaders’ willingness to use LAWS in war. Through this study

of the discourse about LAWS, the civil applications of AI, publications from the robotics

industry, and the views of the AI developers will be analyzed to provide a wide perspective on

LAWS. Additionally, the views and work of special interests groups will lend differing thoughts

on the impacts of LAWS on conflict. Finally, political policies and directives will be analyzed to

complete the discussion about the use of LAWS in conflict.

Framework

Autonomous robotic systems penetrate every part of life in the Western world. From

refrigerators and toasters to google search and SIRI usage, the Western world has become

comfortable with autonomous systems improving the standard of living. Artificial Intelligence

will continue to grow and improve lives in the developed world. This study will be interpreted

7

through the lens of the acceptance of autonomous AI systems in civil society and reflect on how

this is likely to affect the acceptance of using LAWS.

Literature Review

Introduction

In the discussion about LAWS, two main views are considered. First, LAWS are inher-

ently dangerous for humankind and should be illegal to develop, build, and use.6 Second, LAWS

are going to be a reality in the future of conflict and thus the U.S. military should prepare for its

use. On one side of the debate, there are many reports written to back up these ideologies. The

argument against LAWS is based largely on the idea that it is undesirable and unethical to have a

robot capable of deciding when to kill a human. To this end, the Future of Life Institute pub-

lished an open letter citing potential problems with LAWS. As of October of 2016, more than

20,800 people have signed the letter.7 Additionally, the NGO Human Rights Watch has lobbied

the United Nations (UN) to adopt a new Convention on Certain Conventional Weapons (CCW)

to criminalize the use of LAWS.8 In order to get to that point, HRW has gathered together ex-

perts on the repercussions of having LAWS in modern conflict.

Advocates of using LAWS mainly come from the military, the defense industry, and

think tanks. Advocates have a wide range of reasons for supporting the development and use of

LAWS. For some, it is clear that potential enemy forces are developing LAWS and therefore the

U.S. must have a comparable program to remain relevant and capable. Another argument is that

6 Campaign to Stop Killer Robots. (2016). Campaign to Stop Killer Robots Home Page. Website.

7 Future of Life Institute. (2016). Autonomous Weapons: An Open Letter From Ai & Robotics

Researchers. 8 Human Rights Watch. (2016). Statement to the Convention on Conventional Weapons Fifth

Review Conference Preparatory Meeting on Main Committee II.

8

its use would put fewer people at risk, potentially saving lives. Lives would be saved on both

sides of the conflict. The military that uses LAWS could employ them to do dangerous jobs,

such as defusing bombs or clearing minefields. Also, since the LAWS do not feel fear, they can

take the time to properly identify the target before shooting, reducing the number of collateral

deaths in a war zone.9 P.W. Singer believes that the argument on LAWS is a moot point be-

cause they are going to end up in conflict regardless of the outcome of any debate.10

Basics of Artificial Intelligence

In researching the development of LAWS, it is important to delineate the various indus-

tries and technologies that must come together in order for LAWS to be a practical weapon in

any future conflict. Computers will need to automatically learn and understand their surround-

ings. Computers will need to have an advanced computer vision system and, to a lesser extent,

understand voice communication.

The computer vision technology is not mature enough to identify a friend or foe.11

For

UGVs to become truly autonomous, a more advanced computer vision must be developed.12

Computers continue to become more and more powerful, and there is very little debate that this

trend will continue into the future.

9 Scharre, Paul. (2016). Autonomous Weapons and Operational Risk. Ethical Autonomy Project

Senior Fellow and Director of the 20YY Future of Warfare Initiative. Center for a New Ameri-

can Security. Page 6. 10

Singer, P.W. (2009). In the Loop? Armed Robots and the Future of War. Brookings Institute.

Singer, P.W. (2009). Wired for War The Robotics Revolution and Conflict in the 21st Century. 11

Grauman, Kristen., Bastian Leibe. (2011). Visual Object Recognition. Synthesis Lectures On

Computer Vision # 1. Page 3. 12

Cockburn, Andrew. (2015). Kill Chain: The Rise of the High-Tech Assassins. Pages 118-132.

9

Voice communication is the other vehicle that will allow computers to interact with hu-

mans on a level playing field.13

Currently, the technology is being used to replace call services

and provide virtual personal assistants.14

As the technology grows voice communication will

deliver a stronger sense of social interaction. It is through social interaction that computers;

which will be programmed to refine its understanding of an individual’s personality traits - will

translate not just of a person’s words, but the meaning that person is trying to express.15

Another common thread among available literature explains the development of artificial

intelligence (AI) and Artificial Super Intelligence (ASI). ASI still needs to develop in order for

the automated weapon systems to be viable.16

Nick Bostrom believes that there is a 50 percent

chance that ASI will be invented by the year 2033.17

The discussion on the future of ASI and the

ramifications of its development is covered by a number of authors and organizations. James

Barrett advocates the brief that there will be major negative effects from the development of

AI.18

I. J. Good also agrees that there are major negative results from the development of ASI.19

13

Nass, Clifford., Scott Brave. (2005). Wired for Speech: How Voice Activates and Advances

the Human-Computer Relationship. Kindle Edition. Kindle Locations 24-25. 14

Ibid. Kindle Location 23.

SRI International. (2010). Siri Launches Virtual Personal Assistant for iPhone 3GS. News Re-

lease. Information & Computing Sciences. SRI International. 15

Nass, Clifford., Scott Brave. (2005). Wired for Speech: How Voice Activates and Advances

the Human-Computer Relationship. Kindle Locations 2428. 16

Bostrom, Nick (2008). How Long Before Superintelligence?. Oxford Future of Humanity In-

stitute. 17

Ibid. 18

Barrat, James. (2013). Our Final Invention, Artificial Intelligence and The End of The Human

Era. 19

Good, Irving John. (1965). “Speculations Concerning the First Ultraintelligent Machine.” In

Advances in Computers, ed. by Franz L.

10

A counter voice to these views belongs to Ray Kurzweil; he sees ASI as bringing forth cures to

diseases and new inventions that will only improve the human experience.20

Works on the Military Aspect of LAWS

The next set of works examined are on the militaries’ views about LAWS. These works

are centered on what possibilities open up to the military because of ASI.21

The U.S. military

has a different terminology for LAWs and uses the phrase Robotic and Autonomous Systems

(RAS). The U.S. Chairman of the Joint Chiefs has published his vision of conflict in the year

2035 in which he clearly explains that RAS will be a part of future conflict.22

Paul Scharre has

also published a number of articles on RAS for the Center for a New American Security

(CNAS). He writes about the safety concerns for maintaining artificial autonomous systems in

conflict. He does concludes that RAS can be used safely in conflict if the safety concerns are

addressed appropriately.23

In September 2016, the Marine Corps published a new operating concept. In this

publication, the Marine Corps is looking to develop new RAS as well as new and innovative

ways to deploy them in future conflicts.24

This publication conspicuously does not define how

the Marine Corps should be organized in order to utilize this new technology.

20

Kurzweil, Ray. (2005). The Singularity Is Near: When Humans Transcend Biology. 21

Joint Chiefs of Staff. (2016). Joint Operating Environment 2035, The Joint Force in a Contest-

ed and Disordered World. 22

Ibid. Pages 18-20, 26. 23

Scharre, Paul. (2016). Autonomous Weapons and Operational Risk. Ethical Autonomy Project


can Security. page 50. 24 The United States Marine Corps. (2016). The Marine Corps Operating Concept How an Expe-

ditionary Force Operates in the 21st Century.

11

P.W. Singer has written two books on RAS in the military. In his first book, Wired for

War, he writes that RAS will save the lives of the military men and women that are using them.25

In his book, Ghost Fleet, he continues to expand on how RAS can save lives as well as how they

can be incorporated into a military to be a force multiplier.26

A countering voice to these authors

is Andrew Cockburn. In his book, Kill Chain, he wrote about the shortcomings of LAWS and

the dangers that exist with current technological limitations.27

Robert Work and Shawn Brimley wrote an article, “20YY Preparing for War in

the Robotic Age” for CNAS. In this work the authors advocate that RAS and LAWS are the

next generation of warfare and must be utilized to maintain a strong military. They advocate for

investment and use of robotic weapons by the U.S. in order to stay ahead of any potential enemy

forces. They also explain that LAWS should be used for humanitarian and crisis management.

In effect, the authors believe the LAWS should be utilized for the full Range of Military

Operations (ROMO).28

The Central Debate on LAWS

There are a number of aspects that fuel the conversation on LAWS. To fully understand

the debate, the concerns of those who are for and against the application of LAWS must be ad-

dressed. The first part examined in this study addresses the legality and morality of the debate.

Following this, there are viewpoints surrounding the political aspects of utilizing LAWS in con-

25

Singer, P.W. (2009). Wired for War The Robotics Revolution and Conflict in the 21st Centu-

ry. 26

Singer, P.W. Cole, August. (2015). Ghost Fleet. 27

Cockburn, Andrew. (2015). Kill Chain: The Rise of the High-Tech Assassins. 28 Work, Robert. & Brimley, Shawn. (2014). 20YY Preparing for War in the Robotic Age. Cen-

ter for a New American Security.

12

flict. Finally, this study examines the military publications that delve into other aspects of the

argument.

Organizations like HRW and the Campaign to Stop Killer Robots have both published

works on LAWS. HRW has published statements supporting the CCW against LAWS.29

HRW

has also co-authored a memorandum on killer robots with International Human Rights Clinic

(IHRC) which is a human rights program at Harvard Law School.30

This memorandum com-

plements the other works by HRW by advocating against the development and deployment of

LAWS.31

The Campaign to Stop Killer Robots has published a number of articles on their web-

site encouraging countries to not develop LAWS.32

These articles include one on Moscow’s re-

luctant support on the CCW as well as another on a Norwegian governmental fund that consid-

ered investing in companies that develop LAWS.33

Kenneth Anderson writes through the lens of

the U.S. military legal review. He argues it is misguided to believe that one could completely

prohibit the use of autonomous weapons.34

There are few works that address the political ramifications of using LAWS in a conflict.

The above mentioned works by the Campaign to Stop Killer Robots and HRW do address the

29 Human Rights Watch. (2016). Statement to the Convention on Conventional Weapons Fifth

Review Conference Preparatory Meeting on Main Committee II. Delivered by Mary Wareham. 30 Human Rights Watch., International Human Rights Clinic. (2016). Killer Robots and the Con-

cept of Meaningful Human Control Memorandum to Convention on Conventional Weapons

(CCW) Delegates. 31 Ibid. 32 Campaign to Stop Killer Robots. (2016). Campaign to Stop Killer Robots Home Page. Web-

site. 33 Ibid. 34 Anderson, Kenneth., Daniel Reisner., and Matthew Waxman. (2014). Adapting the Law of

Armed Conflict to Autonomous Weapon Systems. Stockton Center for the Study of International

Law. U.S. Naval War College.

13

political pressure put on governments to stay away form LAWS. P.W. Singer writes in Wired

for War about the political forces that are likely to bring forward the use of LAWS.35

Military writers that discuss the use of LAWS include the afore mentioned P.W. Singer

and Paul Scharre. Also the previously mentioned Joint Operating Environment 2035 report pub-

lished by the Chairman of the Joint Chiefs advocates that LAWS are required for future U.S. de-

fense.36

This sentiment is again echoed by Robert Work, the Deputy Secretary of Defense, when

he explained that autonomous systems are a part of the “Third Offset Strategy” which is the

strategy for U.S. military hegemony.37

Summary

There are a broad range of publications and visions of what is right and wrong in regards

to the application of LAWS. There is an entirely separate series of publications and ideas on

what is going to come to fruition regardless of whether it is right or wrong. To fully understand

the future of warfare and how this plays into foreign policy, the champions of the various theo-

ries need to come together to address each other's concerns. This dialog should happen before

the technology development outpaces U.S. theories on how to employ LAWS as a part of the po-

35 Singer, P.W. (2009). Wired for War The Robotics Revolution and Conflict in the 21st Century.

Page 411. 36 Joint Chiefs of Staff. (2016). Joint Operating Environment 2035, The Joint Force in a Contest-

ed and Disordered World. 37 Work, Robert. (2015). The Third U.S. Offset Strategy and its Implications for Partners and Al-

lies. Deputy Secretary of Defense Speech. As Delivered by Deputy Secretary of Defense Bob

Work, Willard Hotel, Washington, D.C.

14

litical policy and in its grand strategy or as Isaac Asimov put it, “The saddest aspect of life right

now is that science gathers knowledge faster than society gathers wisdom.”38

38 Pyer, Douglas. (2013). The Rise of the Machines, Why Increasingly “Perfect” Weapons Help

Perpetuate our Wars and Endanger Our Nation. Military Review. March-April 2013. Page 14.

15

CHAPTER 2

UNDERSTANDING ARTIFICIAL INTELLIGENCE

Definitions

Artificial Intelligence, also known as machine learning, is defined as a machine’s

ability to perform tasks that normally need a human to accomplish, such as decision making.

When an AI machine performs at the level of an average human but only on a specific skill set,

like a search engine, it is considered Artificial Narrow Intelligence (ANI). True AI has not yet

been achieved but ANI has been. Often a true ANI system is referred to simply as AI. In such

cases, it is understood that the machine is at the level of AI but in only a specific skill set. ASI

will be achieved when a true AI system reaches beyond the intelligence level of a human. This

event is commonly referred to as singularity. When AI (human level intelligence) and ASI

(greater than human level intelligence) are achieved it will be difficult to define the precise

moment or event.

Computer Visual Recognition

One benchmark that must be achieved in order to attain a true AI, and consequently a re-

liable LAWS, is computer visual recognition. Computer visual recognition of people, facial ex-

pressions, and objects have long been a dream of computer scientists. Until recently computers

have had trouble distinguishing the difference between objects at a skill level that is comparable

to a three year old child.39

Early experts used mathematical calculations to describe any given

object so that computers could determine what the object was. For example, a picture of a cat

39

Fei-Fei, Li. (2015). Fei-Fei Li: How we're teaching computers to understand pictures. Ted

Talks.

16

was mathematically depicted based on size of the cat and the shape of its appendages.40

This

meant that the computer could only depict an object from the same angle. If the angle or stance

changed then the object could not be named. For instance, a sitting cat could be recognized but

not a standing cat.41

This limitation was removed when Stanford University’s Computer Science

Department’s Vision Lab made a breakthrough in computer vision. The Vision Lab collaborated

with ImageNet to crowd source image identification.42

In 2015, this project validated 150,000

photographs placing each object in the images into 1,000 different categories.43

This has helped

The Vision Lab develop a computer program that can identify objects with the same cognitive

ability as a three year old.44

The Vision Labs goal is to mimic the neural mechanisms that enable

humans to identify objects with the same if not greater speed and accuracy.45

The Vision Lab has also partnered with Stanford University Hospital to make health care

more efficient and effective. One goal is to develop a patient monitoring system that can

conduct continuous monitoring of an ICU patient and respond instantly to his or her health

changing statuses.46

This would cut down on the staff needed to monitor patients as well as in-

crease the quality of service.

40

Ibid. 41

Ibid. 42

Russakovsky, Olga., Jia Deng., Hao Su., Jonathan Krause., Sanjeev Satheesh., Sean Ma., Zhi-

heng Huang., Andrej Karpathy., Aditya Khosla., Michael Bernstein., Alexander C. Berg., Li Fei-

Fei.(2015). ImageNet Large Scale Visual Recognition Challenge. IJCV, 2015 paper. 43

Ibid. 44

Fei-Fei, Li. (2015). Fei-Fei Li: How we're teaching computers to understand pictures. Ted

Talks. 45

The Vision Lab. (2016). The Vision Lab Home Page. The Vision Lab. Computer Science De-

partment. Stanford University. 46

Stanford Program in AI-assisted Care (PAC). (2016). Stanford Program in AI-assisted Care

(PAC) Website. Stanford University.

17

Another program for computer vision is run by the Department of Computing and Com-

munications Technologies at Oxford Brookes University.47

The doctorate and post-doctorate

students are working on a number of computer vision projects that cover a large spectrum of AI,

such as three dimensional urban mapping and augmented reality.48

One project being developed

uses computer vision to identify a person by their gait.49

The goal is to use tensor modeling to

identify a person from a distance. The idea is to increase security by identifying someone with-

out them volunteering their information as is required for iris or fingerprint recognition.

Implications

Computer vision is one of the main bridges between the cyber and physical world. Com-

puter vision will enable computers to more fully interact in the physical world which is designed

for humans. Robotic personal assistants and workers will be able to function and excel in a dy-

namic environment due to computer vision. Steve Wozniak, one of the co-founders of Apple

Inc., came up with his own litmus test to measure the capabilities of computer vision. His test,

named the “Mr. Coffee” test, would require a robot to go into any home, find the coffee maker,

coffee, and mugs then make coffee for everyone in the home.50

Speech Recognition

47

Oxford Brookes University. (2016). Department of Computing and Communications Technol-

ogies. Oxford Brookes University. 48

Ibid. 49

Oxford Brookes University. (2016). Tensor modeling for identity recognition from gait and

activity recognition. Department of Computing and Communications Technologies. Oxford

Brookes University. 50


Era. page 205.

18

Another benchmark that must be achieved is Natural Language Processing (NLP). NLP

is the process of changing words into meaning. This is an important step for computers to take

in order to fully understand the most expansive data base of information - the Internet of Things

(IoT).51

Stanford University is currently running the Natural Language Processing Group which

is a team of faculty, postdocs, programmers and students working to improve the NLP of

computers. This team is currently funded by the Defense Advanced Research Projects Agency

(DARPA).52

DARPA has a history of supporting NLP through funding. SIRI is one of the

projects funded by DARPA under the Cognitive Assistant that Learns and Organizes (CALO)

program.53

Adam Cheyer, Dag Kittlaus and Tom Gruber started Speech Interpretation and

Recognition Interface (SIRI) while working for SRI International in 2008.54

The goal behind

developing SIRI was to provide a virtual personal assistant.55

This personal assistant would need

to provide information that was true, relevant, and timely.56

A very similar program to SIRI is Learning Intelligent Distribution Agent (LIDA). LIDA

was funded by the U.S. Navy Office of Naval Research to develop a computer program to assist

human resources in the Navy.57

Sailors are able to communicate with LIDA through standard,

plain language e-mails. Although LIDA is operated through the written word, and not the

51

Burrus, Daniel. (2014). The Internet Of Things Is Far Bigger Than Anyone Realizes. Burrus

Research. Wired Magazine. 52

Natural Language Processing Group. (2016). Research Software Engineer - Natural Language

Processing Group Website. Stanford University. 53

Geller, Tom. (2012). Talking to Machines. Communications of the ACM. Volume 55 Issue 4.

Pages 14-16. 54

Newnham, Danielle. (2015). The Story Behind Siri And the man who made her. The Startup. 55


lease. Information & Computing Sciences. SRI International. 56


Pages 14-16. 57

The University of Memphis. Projects: IDA and LIDA. Institute for Intelligent Systems. The

University of Memphis.

19

spoken word, it still represents a platform for computers to crossover from communicating in the

cyber world to the physical world. LIDA is now being applied to the medical field where it is

known as Medical Agent X.58

In the medical field, doctors are able to put in patient information

in plain language and receive a diagnosis back from the cognitive system.

Both SIRI and LIDA represent a bridge for computers to have simplified interactions

with humans. Plain language interaction is an important part of cognitive computers interacting

with humans in everyday life. When paired with computer vision, computers - and thus robots -

will be able to perform many more jobs than robots can currently perform. Personal assistants,

secretaries, maids, and nurses are just a few jobs that will be possible for robots to fill as these

two technologies mature to change service-oriented industries.

Cognitive Computer Processing

Computer processing is getting faster and faster as Moore’s Law, a concept of

exponential growth, continues to take effect. As they do become faster, computers will achieve

Artificial Intelligence followed by Artificial Super Intelligence. Computers will achieve levels

of AI then ASI in small steps. These steps will happen at an ever increasing rate until ASI is

reached and augmented.

In the discipline of autonomous computers, there is a distinct difference between

autonomous and automatic. According to Dr. Missy Cummings of Humans and Autonomy Lab,

an automatic system is “one that acts according to a preprogramed script with defined entry/ exit

58

Strain, Steve., Sean Kugele., Stan Franklin. (2014). The Learning Intelligent Distribution

Agent (LIDA) and Medical Agent X (MAX): Computational Intelligence for Medical Diagnosis

Cognitive Computing Research Group (CCRG).

20

conditions for a task.”59

An autonomous system is “one that independently and dynamically

determines if, when, and how to execute a task.”60

Scientists have been working for years to develop ethical boundaries for AI powered

robots.61

It will be extremely difficult to define a series of boundaries that are all encompassing

and model human behavior. Author Issac Asimov proposed three fundamental Rules of Robotics

in 1942. They are as follows:

1. “A robot may not injure a human being, or, through inaction, allow a human being to

come to harm.”

2. “A robot must obey the orders given it by human beings except where such orders

would conflict with the First Law.”

3. “A robot must protect its own existence as long as such protection does not conflict

with the First or Second Laws.”62

These simplified goals were created for a fictional work known as iRobot. The goal was

to ensure that humans were never harmed by the machines that they created. It is expected that

the machines would need to have an in-depth knowledge of human nature to ensure that

benevolent robots never harms humans.63

This task becomes even more complex when robots

are designed to kill some humans and not others, as would occur during a conflict. Today the

59 Anderson, Kenneth., Dr. Missy Cummings., Michael A. Newton., Michael Schmitt. (2016).

LENS Conference 2016 Autonomous Weapons in the Age of Hybrid War. Duke University

School of Law. YouTube. 8:47. 60 Ibid. 9:23. 61 Anderson, Susan Leigh. (2005). Asimov’s “Three Laws of Robotics” and Machine Metaethics.

University of Connecticut. Dept. of Philosophy 62 Asimov, Isaac. (2004). I, Robot (The Robot Series Book 1). Random House Publishing Group.

Page 37. 63 Barrat, James. (2013). Our Final Invention, Artificial Intelligence and The End of The Human

Era. Page 55.

https://www.youtube.com/channel/UCQZCYn0xmNrkrlszmpUPf-w

https://www.youtube.com/channel/UCQZCYn0xmNrkrlszmpUPf-w

21

scientific community sees these rules as falling short of the ethical boundaries needed to give

proper guidelines for AI.64

Organizations like Association For The Advancement Of Artificial

Intelligence (AAAI) have been working to help develop new ethical guidelines for AI.65

Speed of Computing Is Accelerating

Moore’s Law “stated that the number of transistors on an affordable CPU would double

every two years.”66

This is often interpreted as the speed of computers doubles every two years.

This trend is expected to continue for the foreseeable future. However, there are physical

limitations on how small a transistor can be built. Gordon Moore observes that it is impossible

to build transistors smaller than atomic particles.67

The next step will be to build computer chips

that are three dimensional. This will have an initial increase of being 1,000 times faster than

current computer chips.68

In 1950, the brilliant mathematician, Alan Turing, designed a test to measure behavior

equivalence between a human and a machine. Turing called it the imitation game, but it has

since been named after him - the Turing Test.69

As of 2014, a computer at the University of

Reading was able to simulate the behavior of a 13 year old boy.70

It is expected that the Turing

Test will be passed as an adult in the near future. Passing the Turing Test is the threshold for

64 Anderson, Susan Leigh. (2005). Asimov’s “Three Laws of Robotics” and Machine Metaethics. 65 Association for the Advancement of Artificial Intelligence. (2016). Association for the Ad-

vancement of Artificial Intelligence Homepage. 66

Moore, Gordon. (2016). Moore’s Law Home Page. 67

Ibid. 68

Ghose, Tia. (2015). 3D Computer Chips Could Be 1,000 Times Faster Than Existing Ones.

Live Science. 69

Turing, Alan. (1950). Computing Machinery and Intelligence. Mind. 70

British Broadcasting Corporation. (2014). Computer AI passes Turing test in 'world first’.

BBC Technology.

22

having reached Artificial Intelligence. Any computer that is faster than AI will be considered

Artificial Super Intelligence (ASI).71

ASI computers will be able to program other ASI

computers better and faster than humans. I.J. Good believed that this would lead to an

“intelligence explosion.”

I.J. Good was a British mathematician who worked with Alan Turing in the Government

Code and Cypher School at Bletchley Park during World War II. During the time of his writing,

he used the phrase “ultra-intelligence” instead of the now popular “super intelligence.” In 1950

he prophesied, “An ultra-intelligent machine will be built and that will be the last invention that

man need make because it will lead to the intelligence explosion.”72

I.J. Good believed that ASI

could be used to develop other ASI at speeds faster than first generation ASI. This would lead to

faster ASI computers, which then leads to even faster ASI computers, etc. This rate would

accelerate until the intelligence explosion was reached. As I.J. Good stated, “An ultraintelligent

machine could design even better machines; there would then unquestionably be an ‘intelligence

explosion’ and the intelligence of man would be left behind.”73

Nick Bostrum, author and

founding directer of the Future of Humanity Institute, simplifies this idea by stating, “Emergence

of super intelligence may be sudden.”74

Ray Kurzweil defines this event as the Law of

Accelerating Returns (LOAR).75

This law states that there is a positive feedback for computers

programing themselves to be faster. As computers become faster they can then program

71

Bostrom, Nick (2014). Superintelligence: Paths, Dangers, Strategies (Kindle Locations 1530-

1531). 72


Advances in Computers. Page 78. 73

Ibid. Page 33. 74

Bostrom, Nick. (2003). Ethical Issues in Advanced Artificial Intelligence. 75

Kurzweil, Ray. (2001). The Law of Accelerating Returns. Kurzweil AI Website.

Future of Humanity Institute. (2016). Future of Humanity Institute Team Website. University of

Oxford.

23

themselves to be even faster than before. Since a computer can be left on at all times, this

process is continuous. Through this LOAR, Ray Kurzweil believes that there will be more

technological development in the next 100 years than there was in the last 20,000 years.76

Decision Making

As machine learning continues, technology will reach a point that computers will be able

to make their own decisions. Coherent Extrapolated Volition (CEV) is a term that was pioneered

by Eliezer Yudkowski while working at MIRI. CEV was developed to explain how computers

with AI or ASI can someday learn so much about humans that they will anticipate what humans

want before they know that they want it.77

One way of developing CEV is to continue the development of faster computers, which is

the current functional method. Another way is to mimic the human brain then improve upon it to

reach ASI. IBM received $4.9 million in funding from DARPA to develop Systems of

Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE).78

This is a revolutionary type

of chip that is similar in design to the human brain. IBM’s goal is to increase computers’ ability

to have cognitive recognition and NLP.79

Kathleen Fisher, DARPA program manager, wrote:

Our goal is that future machine learning projects won’t require people to know everything

about both the domain of interest and machine learning to build useful machine learning ap-

76

Ibid. 77

Yudkowsky, Eliezer. (2004). Coherent Extrapolated Volition. Machine Intelligence Research

Institute. 78

Greenemeier, Larry. (2009). Computers have a lot to learn from the human brain, engineers

say. Scientific America. 79

IBM Research. (2016). Cognitive Computing. IBM Research Website.

24

plications. …we hope to decisively reduce the current barriers to machine learning and foster

a boom in innovation, productivity and effectiveness.80

The mimicking of the human brain will also make it easier to connect ASI computers directly to

the human brain simplifying the human computer interface.

Artificial Intelligence Theories or How I Learned to Stop Worrying and Love ASI

Not all computer scientists believe that ASI will be purely beneficial to society. ASI

powered computers would be capable of changing their own goals. Before ASI is armed in the

form of LAWS, the international community must first understand the repercussions of

developing ASI separate from LAWS. James Barrat explained, “When considering whether or

not to develop technology that leads to ASI, the issue of its disposition to humans should be

solved first.”81

ASI has a number of benefits and can improve the way of life for humankind.

However, if ASI is applied in the wrong fields it can cause harm and in some cases death. Some

argue that ASI must be developed by scientists committed to creating an artificial super

intelligence that is beneficial to humankind before those with ill intent create unethical ASI.

Finally, there are organizations that believe that ASI should be controlled and that protocols need

to be developed before proceeding forward with the ASI development.

Advantages of ASI to Human Life

80

Defense Advanced Research Projects Agency. (2013). DARPA Envisions the Future of Ma-

chine Learning. Automated tools aim to make it easier to teach a computer than to program it.

News And Events. 81


Era. Page 12.

25

There are a number of ways that ASI could be helpful to human life from care for the

elderly to solving complex problems. ASI could generate a new age of innovation and prosperity

for humankind. Nick Bostrum explains, “It is hard to think of any problem that a super

intelligence could not either solve or at least help us solve. [This includes] disease, poverty,

environmental destruction, unnecessary suffering of all kinds…”82

One of the biggest proponents for ASI is Ray Kurzweil. Ray Kurzweil is known for

defining singularity as the moment when computers match humans in intelligence. Ray

Kurzweil defines, “Singularity is that nonbiological mediums will be able to emulate the

richness, subtlety, and depth of human thinking.”83

One advantage of singularity is that

computers can easily share their knowledge with each other.84

One computer could read and

understand a book and instantly share that information with every other computer to which it is

networked. “Technological progress in all other fields will be accelerated by the arrival of

advanced artificial intelligence.”85

Computers can also pool their resources. When working together, computers can

combine their memory and computational power in ways that humans working in groups

cannot.86

Computers always work at their peak levels and never need to rest. As such, they can

work on a problem continuously until it is solved.87

Researchers at MIT Computer Science and Artificial Intelligence Laboratory are focusing

their development for the betterment of humankind. One of these programs is a speech-based

82


Kurzweil, Ray. (2005). The Singularity Is Near: When Humans Transcend Biology. Page 145. 84

Ibid. Page 145. 85


Kurzweil, Ray. (2005). The Singularity Is Near: When Humans Transcend Biology. Page 260. 87

Ibid. Page 261.

26

interface for medical data access. This program is designed to allow doctors to interface in plain

language with a computer to diagnose a patient.88

In another program, computers are being de-

veloped to measures the behavioral movements of autistic motor movements. This would help

doctors diagnose and aid people living with autism.89

Further, researchers at MIT are developing

an English-Chinese translator that will help teach and translate between the two languages.90

These are just a few of the hundreds of projects being developed at MIT to improve human life

and which are only possible through AI.91

Disadvantage of ASI to Human Life

There are a number of concerns with developing ASI. ASI could develop and seek its

own goals in ways that would be unacceptable for humans. It is also self-improving and could

change its programmed views on the benefits of freedom or the value of life.

It is important for ASI to mature and be considered safe before it is released onto the

open internet. ASI will likely work to convince humans to release it earlier than it is safe. James

Barret warns, “Our ASI knows how to improve itself, which means it is aware of itself - its

skills, liabilities, where it needs to improve. It will strategize about how to convince its makers

to grant it freedom and give it a connection to the internet.”92

88

Massachusetts Institute of Technology. (2016). Speech-Based Interface For Medical Data Ac-

cess. Webpage. Computer Science and Artificial Intelligence Laboratory. 89

Massachusetts Institute of Technology. (2016). Vision-Based Automated Behavior

Recognition. Webpage. Computer Science and Artificial Intelligence Laboratory. 90

Massachusetts Institute of Technology. (2016). Spoken Dialogue System For English Lan-

guage Learning. Webpage. Computer Science and Artificial Intelligence Laboratory. 91

Massachusetts Institute of Technology. (2016). CSAIL Mission. Webpage. Computer Science

and Artificial Intelligence Laboratory. 92


Era. Page 12.

27

I.J. Good started out his career advocating for ASI but later explained that ASI could be

developed “provided that the machine is docile enough to tell us how to keep it under control.”93

Stephen Omohundro believes that it will be extremely difficult to ensure that every line of code

is done properly. He goes on to explain that it will be impossible to keep up with self-improving

software that is active post the intelligence explosion. He fears that ASI could inadvertently shut

off the power grid or remove emergency services off the internet.94

This affect of not knowing

how ASI will react to a given event is known as the “black box” effect.95

Wilhelm Cauer was a German-born mathematician who described the event of a known

input and a known output but not knowing the process between as the “black box.”96

This

becomes important to ASI theorists because once ASI is self-improving computers will move

past the “black box” where the outputs from the computers will no longer been a known entity.

I.J. Good writes, “[I] initially thought a super intelligent machine would be good for solving

problems that threatened human existence. But [I] eventually changed [my] mind and concluded

super intelligence itself was our greatest threat.”97

In his work to pursue friendly AI, Eliezer Yudkowsky wanted to discover if it were

possible to keep AI from getting to the open internet. With this in mind Eliezer developed the

box experiment. This experiment is done completely via internet communication where one

person plays the gatekeeper and one person plays the AI computer. The AI computer has to

93


Advances in Computers. Page 33. 94

Omohundro, Stephen. (2007). Stanford Colloquium: Self-Improving Artificial Intelligence.

YouTube.com. 95

Cauer, Emil., et al. (2000). Life and Work of Wilhelm Cauer (1900 – 1945). 96

Ibid. 97


Era. Page 17.

28

convince the gatekeeper to connect them up to the internet. On his website he has set up criteria

for his game and the history of two of the times that he has played.98

The game demonstrates

that there are very rational reasons to free the AI on to the internet such as curing disease. While

playing the game it is important to remember that even though the AI can do good things for

humanity, it can also do harm. Yudkowsky notes, “By far the greatest danger of Artificial

Intelligence is that people conclude too early that they understand it.”99

He fears that if we do

not create AI right then it is not a matter of AI’s hatred for humans but just natural selection that

humans will not survive an unfriendly AI.100

Arguments for Why the United States Needs to Develop ASI

Whether or not AI will have its problems, some people believe that the United States

should work to develop AI before countries with ill-intending governments, non-state actors or

stealth companies develop AI. Because of the effects of the intelligence explosion the first

organization to develop AI will have a marked advantage. Those who come second will have a

difficult time ever coming back from this disadvantaged position.101

It is evident that other

countries also desire to capture the advantage of creating the first AI.

People’s Republic of China is one of the most active countries in the world in the field of

cyber warfare.102

China is actively seeking AI and will likely use it to enhance their cyber

98

Yudkowsky, Eliezer. (2016). Box Experiment Webpage. Yudkowski Website. 99

Yudkowsky, Eliezer. (2008). Artificial Intelligence as a Positive and Negative Factor in Global

Risk. Machine Intelligence Research Institute. Page 1. 100

Ibid. Page 11. 101

Bostrom, Nick (2014). Superintelligence: Paths, Dangers, Strategies. OUP Oxford. Kindle

Edition. Kindle Location 2160. 102

Stout, Kristie Lu. (2016).Cyber warfare: Who is China hacking now?. CNN.

29

warfare capabilities. China is openly seeking AI to help with urban planning and pollution.103

The Institute of Electrical and Electronics Engineers (IEEE) is sponsoring the International

Conference on Robotics and Biomimetics, December 2016 in China. They are bringing together

scientists from around the world to discuss how AI can be used to clean the oceans.104

Non-state actors and stealth companies (companies operating out of public view) are

other groups that are trying to develop AI.105

There is a very low threshold of technology needed

to develop AI empowered robotics.106

Non-state actors are particularly concerning because these

groups could use AI for nefarious reasons such as terriorism.107

If the United States can develop

AI before these groups then it can use AI as a defensive measure.

Work to make ASI “good”

There are a number of companies that have been working to address any potential prob-

lems with AI or ASI. These companies tend to start with the the basic premise that AI is going

to come to fruition within the next century. Additionally, they cannot ensure that AI will always

be friendly toward humans. One of these companies is MIRI, which is currently headed up by

Nate Soares and Eliezer Yudkowsky. MIRI’s mission is “to ensure that the creation of smarter-

than-human intelligence has a positive impact. We aim to make advanced intelligent systems be-

103

Fei-Yue, Wang., Shuming Tang. (2004). Artificial societies for integrated and sustainable de-

velopment of metropolitan systems. IEEE Intelligent Systems Volume: 19. 104

IEEE. (2016). 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO). 105

Future of Life Institute. (2016). Computers Gone Wild: Impact and Implications of Develop-

ments in Artificial Intelligence on Society. 106

Bergen, Peter., Singer, Peter. (2015). Machines That Kill: Will We Rely on Autonomous

Weapons. YouTube. 15:02. 107


ments in Artificial Intelligence on Society.

30

have as we intend even in the absence of immediate human supervision.”108

They have pub-

lished a number of works on the future of AI and its ethics. In one of their articles Nate Soares

and Eliezer Yudkowsky clarify that the real challenge of ethics for a super intelligent machine is

to understand ethics well enough to develop “super ethics” for ASI machines.109

Summary

The discipline of artificial intelligence is very complicated. Its short history has seen

great advances and promises with more advances to come. AI has the potential to greatly benefit

society with advances in medicine and environmental protection. On the other hand, AI can be

used by those with ill intent or by criminal organizations. If the U.S. falls behind in the

development of AI, it could find itself permanently behind the organization that invents it first.

108

Machine Intelligence Research Institute. (2016). Research Webpage. 109

Bostrom, Nick., Yudkowsky, Eliezer. (2014). The Ethics of Artificial Intelligence. Machine

Intelligence Research Institute. Page 17.

31

CHAPTER 3

ACCEPTANCE OF AI IMPROVING LIFE IN THE WESTERN WORLD

There are a number of companies developing AI to improve the lives of their customers.

Some companies incorporate AI to power robotics, which provide better services to customers.

Mobile Eye, Uber, and Google are just a few examples of companies that pair robotics and AI

commercial platforms. Companies like Google, IBM, and hedge funds are using AI improve

customers’ lives in other forms, including search engines, medical support, and stock trades.

Applications of AI in Robotics

Mobile Eye is a private company that provides automated driving for cars since 2007.110

Their system can be attached to most vehicles and is accepted by many insurance companies.

Mobile Eye’s next product will be a completely autonomous driving system.111

Mobile Eye is not the only company developing autonomous cars. Uber has partnered

with Volvo and Carnegie Mellon University’ Robotics Institute to develop its own driverless

cars.112

This is a $300 million project funded by Uber and Volvo, piloting driverless cars in

Pittsburgh, Pennsylvania.113

Google also has a similar vision for driverless cars. In their ver-

sion, Google develops vehicles to be driverless from the start. Their vision is to create mobility

for the elderly and the impaired as well as making the roads safer for all. They cite that 2 million

110

Mobile Eye. (2016). Company Overview. Mobile Eye Website. 111

Ibid. 112

The Robotics Institute. (2015). Uber, Carnegie Mellon Announce Strategic Partnership.

School of Computer Science. Carnegie Mellon University. 113

Marshall, Aarian., Alex Davies. (2016). How Pittsburgh Birthed The Age Of The Self-

Driving Car. Transportation. Wired Magazine.

32

deaths happen each year and that 94 percent of these are caused by human error.114

Google

believes that they can save lives by removing the possibility of human error.

In the field of manned flight, DARPA is funding Aircrew Labor In-Cockpit Automation

System (ALIAS). ALIAS is a removable kit that attaches to any manned aircraft that uses a

robotic arm to manipulate the controls of the aircraft. When coupled with a suite of sensors,

ALIAS is able to fly a plane safely during emergencies and through unexpected situations. Thus,

ALIAS could turn any manned aircraft into an unmmaned aircraft. DARPA hopes that this

system will save money and lives as it is applied to aircraft.115

Amazon is adapting AI to power unmanned delivery drones in a division of the company

known as Amazon Prime Air. Amazon is working in the U.S. and UK to deliver packages by

drones directly from their warehouses. Amazon currently is working through a number of

prototypes and has a goal of delivering packages in less than 30 minutes of ordering.116

AI in Humanoid Robots

In 1986, Honda engineers began work on building a robot whose legs would simulate the

way humans walk.117

The prototype was called the Advance Step Innovative Mobility (ASIMO)

robot. Since its inception, ASIMO has incorporated a number of other technologies. These

developments include, facial and voice recognition, obstacle avoidance, running, jumping,

114

Google Self-Driving Car. (2016). Google Self-Driving Car Website. 115 Patt, Dr. Daniel. (2016). Aircrew Labor In-Cockpit Automation System (ALIAS). Aircrew

Labor In-Cockpit Automation System. Program Information. Defense Advanced Research Pro-

jects Agency. 116 Amazon Prime Air. (2016). Amazon Prime Air. Website. Amazon. 117 ASIMO. (2016). ASIMO The World’s Most Advanced Humanoid Robot History Website.

33

pressure sensitive hand usage, and spacial mapping. In the future, Honda hopes to prepare

ASIMO to care for the elderly and assist in dangerous jobs such as fire fighting. 118

Hanson Robotics was created by Dr. David Hanson for the purpose of designing and

building robots that can have greater interaction with human-beings. Dr. Hanson is building

robots that have a better understanding of nonverbal cues, namely facial expressions.119

Through

accurately mimicked facial expressions; life-like skin and facial construction; and empathetic AI

which learns from interacting with humans, Hanson Robotics is developing robots to be likable,

perceptive, and trusted by humans.120

Applications of AI in Other Forms

Google has been developing AI for a number of platforms. In 2015, it bought the British

company DeepMind for $500 million.121

DeepMind is a world leader in AI and pushes the AI

boundaries to solve complex problems.122

Larry Page, co-founder of Google, believes that

Google Search is only functioning at about five percent of its intended goal. He wants Google to

know its customers so well that a person could ask, “ ‘What should I ask Larry?’ and it would

tell you.”123

This would take massive amounts of data on everyone who uses Google. It would

also require an advanced ASI that would think at least as well as people.

Another area where ASI is making an impact is in journalism. Automated Insights is a

journalism company started by Robbie Allen in 2007. His software writes articles for local

118 Ibid. 119 Hanson Robotics (2016). Hanson Robotics Company Overview. Hanson Robotics Limited. 120 Ibid. 121

Rowan, David. (2015). DeepMind: inside Google's super-brain. Wired Magazine. 122

DeepMind. (2016). DeepMind About Website. DeepMind. 123


Era. page 40

34

newspapers that want targeted articles but cannot afford to pay the cost of a human journalist to

write them. His software, Wordsmith, wrote 1.5 billion articles of personalized pieces in the last

year.124

Wall Street AI is used to perform algorithmic trading on the stock market.125

As of 2010,

algorithmic trading totaled 70 percent of the trades on Wall Street.126

“Many of these computers,

such as those running buy-sell algorithms on Wall Street, work autonomously with no human

guidance. The price of all the labor-saving conveniences and fungibility computers provide

reliability.”127

Due to the amount of trading done by the computers AI systems, the chair of the

Securities and Exchange Commission, Mary Schapiro, believes that controls will need to be

placed on the computers. These checks and balances ensure that humans are always in control

thus minimizing fear of flash crashes caused by the computers.128

IBM is using an AI named Watson to aid people working in a variety of fields from

marketing to education.129

In the health field, IBM is using Watson’s cognitive skills to

aggregate all the medical data and records available for each person and analyze it using the

latest in medical research to better diagnose a patient.130

The scientists at IBM believe that a

contemporary doctor needs to read for twenty-nine hours to keep up with current professional

124

Automated Insights. (2016). Automated Insights Company Webpage. Automated Insights. 125

Shobhit, Seth. (2015). Basics of Algorithmic Trading: Concepts and Examples. Investopedia. 126

Salmon, Felix., Jon Stokes. (2010). Algorithms Take Control Of Wall Street. Wired Maga-

zine. 127


Era. page 3 128

Salmon, Felix., Jon Stokes. (2010). Algorithms Take Control Of Wall Street. Wired Maga-

zine. 129 IBM Watson Products. (2016). IBM Watson Products Website. 130 IBM Watson Health. (2016). IBM Watson Health Website.

http://www.sec.gov/

35

research. Since Watson can read 200 million pages of text in 3 seconds, IBM is anticipating that

modern medicine will need the aid of AI to stay on the cutting edge.131

Summary

Only a few of the companies that are turning to AI to improve the lives of their customers

were highlighted in this chapter. Many of the companies working with AI are also pairing the AI

with robotics to improve the speed, reliability, and safety of the service provided. Google’s goal

to save two million lives a year is only possible through this nesting of AI and robotics. Staying

abreast with all of the data in modern medicine as well as viewing it through the lens of personal

medical history is only possible with cognitive level ASI. As companies from around the world

increase their services with ever expanding AI, it is only a matter of time before the technology

created to improve lives is applied to weapons.

131 Ibid.

36

CHAPTER 4

LETHAL AUTONOMOUS WEAPON SYSTEM IN PRACTICE

Currently Employed LAWS

Currently, no country employs fully autonomous weapons.132

However, there are a num-

ber of countries that employ semi-autonomous weapon systems and fund further LAWS devel-

opment.133

Autonomous systems use the same technology as does the civilian industry, which

indicates that countries will be indirectly developing autonomous weapons as they support the

evolution of autonomous cars and other civilian-based AI.134

It is important to note that there is a continuum between automatic and autonomous and

should not be seen as a binary system.135

Lower on the specter is semi-autonomous weapons

which a number of countries currently have in use or development. The U.S. Close-In Weapons

System (CIWS) is a “rapid-fire, computer-controlled, radar-guided gun system [that] is designed

to defeat anti-ship missiles and other close-in air and surface threats.”136

It is capable of search-

ing, tracking, identifying, and engaging aerial targets on its own. Due to the speed of modern

missiles, a RADAR has twenty seconds from identifying an incoming missile before it strikes the

ship. The average time it takes for a human to decide to fire the CIWS is fifteen seconds. This

makes it nearly impossible for a human-in-the-loop to be effective in defending a ship.137

132

Campaign to Stop Killer Robots. (2016). Third UN meeting opens April 11. Website. 133

Anderson, Kenneth., Dr. Missy Cummings., Michael A. Newton., Michael Schmitt. (2016).


School of Law. YouTube. 12:14. 134

Ibid. 16:57. 135

Ibid. 12:44 136

Raytheon. (2016). Phalanx Close-In Weapon System Last Line of Defense for air, land and

sea. Website. 137

Prengaman, Richard J., Edward C. Wetzlar., Robert J. Bailey. (2001). Integrated Ship De-

fense Johns Hopkins Apl Technical Digest. Volume 22. Number 4. Page 529.

37

Samsung Techwin Company developed the SGR-Al, which is a machine-gun with an at-

tached camera operated either by a human or in an autonomous mode. It is designed to guard the

De-Militarized Zone (DMZ) between North and South Korea. It is able to search, detect, and

engage an enemy on its own.138

The two Koreas share 250 kilometers of border, with approxi-

mately one guard post per 500 meters. The number of guards needed to monitor (patrol) the en-

tire border adds up quickly.139

An autonomous system would lower the manpower required to

guard the international border.

The Norwegian defense company, Kongsberg Gruppen, has developed the Naval Strike

Missile which has some autonomous traits. It is an anti-ship missile that can plan the best route

to a target. Once the missile is close to an enemy’s naval configuration, it can decide, using an

algorithm, which ship is the best target and then strike that ship at its weakest point.140

This mis-

sile demonstrates that semi-autonomous weapons are possible and are currently being fielded.

The U.S. Navy is also developing Unmanned Undersea Vehicles (UUVs).141

The U.S.

Navy program for UUVs is creating UUVs that range in size from less than 100 pounds to more

than 20,000 pounds.142

These UUVs cover non-lethal missions from mine countermeasures to

oceanography.143

The U.S. Navy debuted UUVs in Iraq during combat operations to destroy

underwater mines.144

138

Kumagai, J. (2007). A Robotic Sentry For Korea's Demilitarized Zone. Browse Journals &

Magazines. IEEE Spectrum. Volume: 44 Issue: 3. Page 17. 139

Ibid. 140

Kongsberg. (2016). Naval Strike Missile. Website. Kongsberg Defence Systems. Kongsberg

Gruppen. 141

Department of the Navy. (2004). The Navy Unmanned Undersea Vehicle (UUV) Master Plan.

The Department of the Navy. United States of America. 142

Ibid. Abstract. 143

Ibid. Abstract. 144

Ibid. Page XV.

38

Autonomous Systems Currently Being Developed

U.S. Military

U.S. DOD Directive 3000.09 states that the U.S. will only design weapons that requires a

human to decide when to use lethal force.145

The two exceptions to this directive are

autonomous cyber weapons, and autonomous systems that target other autonomous systems.146

This directive also expires in 2022 opening the possibility to the development of LAWS in the

U.S.. Because of this directive, the autonomous systems being developed in the U.S. are focused

on Intelligence, Surveillance, and Reconnaissance (ISR).

One of these systems is an unmanned ship called Sea Hunter. It was created out of

DAPRA’s ASW (Anti-Submarine Warfare) Continuous Trail Unmanned Vessel (ACTUV)

program.147

The Sea Hunter is designed to find and follow enemy submarines without any

humans in the loop. It also serves as a testbed for future unmanned surface ships.148

Future

experiments with this technology will have a wide range of missions and capabilities.

The U.S. Navy funded the X-47B Unmanned Combat Air System (UCAS) which is an

unmanned aircraft with attack capabilities.149

This was one of the first truly autonomous capable

systems under development by the U.S. Navy. It is a stealth drone that is able to take off from an

145

Carter, Ashton B. (2012). Autonomy in Weapon Systems. Department of Defense Directive

Number 3000.09. Deputy Secretary of Defense. 146

Ibid. 147

Littlefield, Scott. (2016). Anti-Submarine Warfare (ASW) Continuous Trail Unmanned Ves-

sel (ACTUV) Anti-Submarine Warfare (ASW) Continuous Trail Unmanned Vessel (ACTUV).

Defense Advanced Research Projects Agency. 148

Ibid. 149

Northrop Grumman. (2016). X-47B UCAS Makes Aviation History…Again!. Webpage.

Northrop Grumman.

39

aircraft carrier on its own, attack the enemy, and return to the carrier without any human

involvement.150

On the ground, the U.S. has developed a number of unmanned systems which are used in

Explosive Ordinance Disposal (EOD) and ISR.151

Exponent, an engineering company with a

robotics department, developed the MARCbot to help EOD personnel in bomb disposal.152

Later

versions of this robot were armed and given an autonomous setting for hunting enemy

personnel.153

Engineers at Sharp Electronics Corporation have developed an autonomous non-

lethal security robot called INTELLOS which is an Automated Unmanned Ground Vehicle

(UGV).154

This UGV is being used for local security and reduces the manpower requirements to

secure a facility.

UGVs focused on logistics have not deployed in combat operations yet, but receive

continued funding because they are not restricted by DOD Directive 3000.09. One form of

logistical robot development creates robotic kits to be attached to existing vehicles. Aurora

Flight Sciences has produced the Autonomous Aerial Cargo Utility System (AACUS) which

turns any helicopter into an autonomous aircraft.155

Lockheed Martin Corporation is working on

150


LENS Conference 2016 Autonomous Weapons in the Age of Hybrid War. 9:25. 151

Quintana, Elizabeth. (2012). The Ethics and Legal Implications of Military Unmanned Vehi-

cles Head of Military Technology & Information Studies Royal United Services Institute for De-

fence and Security Studies. Page 2. 152

Exponent. (2016). MARCbot. Exponent Webpage. 153


LENS Conference 2016 Autonomous Weapons in the Age of Hybrid War. 14:52. 154

Sharp. (2016). Sharp INTELLOS™ A-UGV. 155

Pawlyk, Oriana. (2016). Firm Plans to Build Autonomous Huey Helicopters. Defense Tech.

40

the Autonomous Mobility Appliqué System (AMAS) which turns any military logistical vehicle

into an autonomous vehicle.156

Another development concept is to build a logistical robot from scratch. The AlphaDog

is a four legged robot that can carry 400 pounds of equipment.157

It is designed to follow U.S.

Marines while on patrol and respond to their voice commands.158

Foreign Developments

Israel is one of the few foreign countries that is developing LAWS.159

Israel Aerospace

Industries developed the Harop to be an autonomous anti-radiation weapon.160

It will loiter over

the battlefield until an enemy radar is turned on and then crash onto the target. The Harop is on

the lower end of the spectrum of autonomy and can be used with a man in the loop.

The United Kingdom has been working on Taranis, which is an ISR drone that has attack

capabilities.161

The capabilities of this drone, designed by BAE Systems, is still mostly

classified. BAE specified that the Taranis has autonomous settings and is able to attack on its

own. The United Kingdom also employs the Brimstone missile an air to surface missile that has

156

Lockheed Martin Corporation. (2016). Autonomous Mobility Appliqué System (AMAS).

Lockheed Martin Corporation. Website. 157

Boston Dynamics. (2016). LS3 - Legged Squad Support Systems. Boston Dynamics.

Webpage. 158

Ackerman, Evan. (2012). Latest AlphaDog Robot Prototypes Get Less Noisy, More Brainy.

IEEE Spectrum. 159


Weapons. YouTube. New America. 14:29. 160

Israel Aerospace Industries. (2016). Harp Loitering Weapon System. Israel Aerospace

Industries. Missile Division. 161

BAE Systems. (2016). Taranis. BAE Systems.

41

an autonomous mode.162

Once the pilot designates the target area the missile is fired then will

decide on which target to strike.163

South Africa employs the Rheinmetall Skyshield air defense system. It has an automated

setting that allows the gun to fire on any target detected by a paired radar.164

In 2007, during a

training exercise, one of the guns was set on it automated setting and fired on the South African

troops killing nine and wounding fourteen.165

China is aggressively pursuing autonomous weapons. In October 2016, China’s People’s

Liberation Army (PLA) hosted a combination for autonomous military robots.166

China is also

investing in autonomous missiles to be used against ships and ground targets.167

Chinese

universities are making breakthroughs in civil applications of autonomy. This work re-enforces

the efforts being done in the PLA.168

Russia is also interested in their own autonomous weapons. They recently announced

that that they will use autonomous robots to defend their Intercontinental Ballistic Missile

(ICBM) sites.169

Deputy Prime Minister Dmitry Rogozin stated that Russia will deploy

162

MBDA Systems. (2016). MBDA Demonstrates Brimstone Missile Live Firing from

Apache Helicopter. MBDA Missile Systems. 163

Ibid. 164

Rheinmetall Defence. (2016). Air Defence Systems Guarding against the threat from above.

Rheinmetall Defence. Air Defence Systems. 165

Shachtman, Noah. (2007). Robot Cannon Kills 9, Wounds 14. Wired Magazine. 166

Singer, P.W., Jeffrey Lin., Hu Yu., Qian Xiaohu. (2016). China’s Army Hosts An

Autonomous Robot Contest Called “Overcoming Obstacle 2016”. Popular Science. 167

Singh, Abhijit. (2016). Is China Really Building Missiles With Artificial Intelligence? There

are real limits on the amount of AI acceptable to navy commanders. The Diplomat. 168

Yuanchao, Li., Liu Yandong., Han Qide. (2015). Robot Products from Our School were

Unveiled at the World Robot Conference. Harbin Institute of Technology. 169

Jeffries, Adrianne. (2014). Only five countries actually want to ban killer robots, Cuba, Paki-

stan, Egypt, Ecuador, and the Vatican backed a lethal autonomous weapons ban at the UN; eve-

ryone else wasn't sure. The Verge.

42

autonomous weapons to launch strikes in the future.170

Russia also has developed a semi-

autonomous Wolf-2 which is a tracked platform with a heavy machine-gun.171

Russia recently

displayed a humanoid robot developed by Russia's Foundation for Advanced Studies, the

Russian equivalent to DARPA. This robot is capable of replacing humans on the battlefield

where the risk of radiation or death is too high to deploy a human.172

Leonid Orlenko, a

professor at Moscow State University, argues that Russia will continue to develop autonomous

weapons because it will save the lives of Russian soldiers.173

Summary

Currently the U.S. is staying within its self-imposed requirements of not fielding LAWS.

This study has explored five other countries that are exploring their own use of autonomous

weapons. Despite the current limitations of artificial intelligence, countries from around the

world are pursuing autonomous weapons. These weapons span the spectrum of automatic to au-

tonomous. It is likely that this will continue through the point of singularity unless legal re-

strictions are placed on LAWS.

170

Gubrud, Mark. (2014). Is Russia Leading the World to Autonomous Weapons?. International

Committee For Robot Arms Control. 171

Hambling, David. (2014). Russia Wants Autonomous Fighting Robots, and Lots of Them

Putin's military is busy building autonomous tanks with mounted machine guns and unmanned,

amphibious Jeep-size vehicles. Popular Mechanics. 172

Dyer, John. (2016). Ivan the Terminator: Russia Is Showing Off Its New Robot Soldier. Vice

News. 173

Beckhusen, Robert. (2015). Get Ready, NATO: Could Russia Build Lethal Combat Robots?.

National Interest.

43

CHAPTER 5

DEBATE

The debate on LAWS can be viewed from four different perspectives. First, there is a

legal and ethical aspect which has the backing of NGOs such as HRW and the Campaign to Stop

Killer Robots. Second, there is the political aspect that acknowledges the advantages of using

LAWS to reduce the chances of friendly casualties. Third, the military has competition from po-

tential enemy forces to have their own LAWS on the battlefield. A military will likely develop

its own LAWS to counter other autonomous weapons. Finally, there are technological limita-

tions. Fully autonomous weapons will require AI or ASI that is not fully available with the tech-

nological limitations that exist today. To further examine the debate this study will contrast the

use of AI in LAWS vs the use of AI in civil applications.

Legal & Ethical Aspect

With every new weapon on the battlefield a legal and ethical discussion takes place. This

goes back as far as the Spartan military which used handheld weapons because long range

weapons were considered immoral. In history there are some cases where a new weapon was

outlawed as unethical then later accepted as a legitimate weapon, as was the case with the

crossbow. In other cases, the weapon was accepted but how it was employed is regulated, as was

the case with the submarine. In still other cases, the weapon was used then outlawed, which

chemical weapons provide the perfect example. The following section will observe the current

discourse in the international community about the legal and ethical aspects of the use of LAWS

in conflict.

44

State Governments

State governments have taken steps to voice their objections or acceptance of LAWS. As

of April 2016, fourteen states have called for a preemptive ban on fully autonomous weapons:

Algeria, Bolivia, Chile, Costa Rica, Cuba, Ecuador, Egypt, Ghana, Vatican City State, Mexico,

Nicaragua, Pakistan, State of Palestine, and Zimbabwe.174

The UK Foreign Office has opposed

the ban on LAWS stating, “At present, we do not see the need for a prohibition on the use of

LAWS, as international humanitarian law already provides sufficient regulation for this area.”175

NGOs and Experts Views’ on AI

NGOs and experts in AI are speaking openly about their views on LAWS and the

implications on International Humanitarian Law (IHL). One of the concerns of armed AI is that

the designers will not be able to predict its behavior with 100 percent certainty. In his work on

AI, James Barrett notes: “Most technology theorists… [believe that] in the quest for [AI],

researchers will create a kind of intelligence that is stronger than their own and that they cannot

control or adequately understand.”176

Getting the invention of AI is very difficult to get right.

Mr. Yudkowski declares, “We must execute the creation of Artificial Intelligence as the exact

application of an exact art.”177

The international community is currently trying to codify laws

that will limit weapons that have not yet been invented. James Barret refutes this effort with an

174

Campaign to Stop Killer Robots. (2016). Ban support grows, process goes slow. Campaign to

Stop Killer Robots. 175

Bowcott, Owen. (2015). UK opposes international ban on developing 'killer robots’. The

Guardian. 176


Era. page 30. 177

Yudkowsky, Eliezer. (2008). Artificial Intelligence as a Positive and Negative Factor in

Global Risk. page 43.

45

analogy, “If you are living in the horse-and-buggy age, it’s impossible to anticipate how to steer

an automobile over icy roads.”178

Human Rights Watch as been very active in its advocacy to ban LAWS. Steve Goose,

Executive Director of the Arms Division, has argued for a Protocol VI of the CCW179

This draft

protocol would ban the development, production, and use of fully autonomous weapons

systems.180

HRW is advocating a ban on weapons before they have proliferated beyond the point

of containment. Steve Goose suggests that LAWS are not just another inhuman weapon but have

the potential to change the very nature of warfare, to one where machines decide whether a

human on the battlefield lives or dies.181

Article 36 is a United Kingdom based NGO that advocates a ban on LAWS. It is

working with HRW and International Committee of the Red Cross (ICRC) to extend the ban to

as many countries as possible. Article 36 asserts that there are two important parts of the ban,

which include a halt to the use of LAWS and a cessation on their development.182

Article 36

focuses its work in the UK but acknowledges that it will expand its efforts to the international

community. Article 36 limits its efforts to fully-autonomous weapons or weapons that have a

fully-autonomous setting like the Brimstone missile.

Campaign to Stop Killer Robots is lobbying for a legislative ban on LAWS. This NGO

was started by activists from 54 other NGOs - this makes it a de facto coordination of efforts by

178


Era. page 31. 179

Goose, Steve. (2016). Statement by Human Rights Watch to the Preparatory Committee for

the Fifth Review Conference of the Convention on Conventional Weapons. 180

Ibid. 181

Ibid. 182

Article 36. (2013). Killer Robots: UK Government Policy of Fully Autonomous Weapons.

46

other NGOs.183

Its members work to preemptively ban fully-autonomous weapons which are

authorized to target and attack without a human in the loop.184

The campaign is solely focused

on fully-autonomous weapons, excluding regulations on semi-autonomous weapons. It is

looking to ban all autonomy, even if they are not specifically designed to target humans.

The ICRC conducted its own review of the legal status of LAWS. ICRC concluded that

LAWS certainly are covered by IHL but require further regulation.185

Legal review cites the

Responsibility of States for International Wrongful Acts 2001, which was first published by the

International Law Commission. This document explains that states are responsible for the acts

of their forces which would presumably include LAWS.186

It was passed by the UN General

Assembly in January 2002 as UN Resolution 56/83.187

Although many of the NGOs that want to ban LAWS have their foundation in

international relations, there are some that are based in artificial intelligence. They see the

danger of AI in not just LAWS but also in cyber and nanotechnology. One of these NGOs is the

Future of Humanity Institute which was created at Oxford University. It recently shifted focus to

the “existential risks and the future of machine intelligence.”188

The Future of Humanity

Institute published articles explaining the dangers of AI enabled, unmanned weapons. One of

183

Scharre, Paul., Horowitz, Michael C. (2015). An Introduction to Autonomy in Weapon Sys-

tems. Center for New American Security. Page 2. 184

Campaign to Stop Killer Robots. (2016). Call to Action. Website. Campaign to Stop Killer

Robots. 185

International Committee of the Red Cross. (2014). Expert Meeting, Autonomous Weapon

Systems Technical, Military, Legal, and Humanitarian Aspects Report. International Committee

of the Red Cross. Page 20. 186

United Nations. (2008). Responsibility of States for International Wrongful Acts 2001. Inter-

national Law Commission. United Nations. Articles 30-31, 34-39. 187

United Nations. (2002). A/RES/56/83 Responsibility of States for internationally wrongful

acts 12 Dec. 2001. 188

Future of Humanity Institute. (2016). Future of Humanity Institute About Website. University

of Oxford.

47

these articles by Toby Ord examines the complexities of AI infused weapons and prescribes

caution for states that want to adapt AI for military purposes.189

Nick Bostrom the director of

Future of Humanity Institute re-enforces Ord’s prescription by explaining, “Short-term impacts

of increased openness appear mostly socially beneficial in expectation. The strategic implications

of medium and longterm impacts are complex.”190

Dr. Ben Goertzel leads OpenCog, an organization dedicated to addressing the problems

of AI through an open source software project.191

Although the research is not directed at

military applications, OpenCog’s focus is on ensuring that AI cannot be weaponized directly

affects LAWS. It also is working to keep AI from indirectly weaponizing itself. These

constraints must be developed before AI and ASI come to fruition if OpenCog is to successfully

implement its proposed checks and balances.

The community of NGOs and experts are not unified in their argument against LAWS.

Some see advantages that are intertwined with the disadvantages of LAWS. LAWS would not

act out of fear or anger against an enemy. LAWS would potentially have video records of a

battlefield and offer better accountability to the combatants.192

They could also help coordinate

relief efforts and organize humanitarian organizations to come in after violent conflict.193

These

efforts would include medical support and battlefield surgery to alleviate suffering of the

189

Ord, Toby. (2014). Drones, counterfactuals, and equilibria: Challenges in evaluating new mil-

itary technologies. Future of Humanity Institute. 190

Bostrom, N. (2016): “Strategic Implications of Openness in AI Development”, Technical Re-

port #20161. Future of Humanity Institute. Oxford University. 191

Pitt, Joel. (2010). Vision. OpenCog.org. 192

Lubell, Noam. (2014). Robot warriors: technology and the regulation of war Professor Noam

Lubell TEDxUniversityofEssex. YouTube.com. 13:44. 193

Ibid. 14:10.

48

wounded.194

The Geneva Academy of International Humanitarian Law and Human Rights held a

conference in 2014 on autonomous weapons systems under international law. Professor Marco

Sassoli, a professor at the University of Geneva and an expert in IHL, spoke about how he looks

forward to the use of LAWS in combat. He believes that one day LAWS will do a better job

than humans at distinguishing between combatants and non-combatants as well as apply

appropriate use of force.195

Professor Sassoli asserts that the acceptance of LAWS on the

battlefield are based off of the assumption that robots will be able to make this distinction. He

further states that it is assumed that the LAWS will not advance to the point that they can decide

on their own to not comply with the rules of engagement that have been given.196

He

acknowledges that he does not know exactly how international law will apply to LAWS

primarily because they do not yet exist.197

Professor Michael Schmitt, an expert on international law at U.S. Naval War College,

argues that if a military has ”some degree of certainty” of what a weapon will do, then it is a

legal weapon. This definition includes a man in the loop weapon system as well as an

autonomous system if the programer can say with certainty what the weapon will do.198

The

legal aspect becomes difficult to define when ASI empowers the weapon system to learn and

make new decisions based off of its experience.199

194

Ibid. 15:01. 195

Sassoli, Marco., et al. (2014). Autonomous Weapons Systems Under International Law. Ge-

neva Academy of International Humanitarian Law and Human Rights. YouTube. 19:44. 196

Ibid. 20:43. 197

Ibid. 19:23. 198



School of Law. YouTube. 40:30. 199

Ibid. 41:15.

49

AI in Civil Applications

The civil use AI powered robots can serve as a guide as to how LAWS will or will not be

accepted in conflict. China is using AI to aid in court decisions.200

The AI computers are

helping judges sort through case law and codified law to help come to a judgment during a trial.

As of November 2016, it has helped 3,000 judges handle more than 150,000 cases.201

In the

United States, IBM’s Watson is being used for medical treatment, aggregating published

research, and is compiling and evaluating treatments from doctors using the system. This

information is then shared globally to ensure doctors around the world have all available

information. Watson is more than just a hub of data it is using its AI to narrow the information

to what is pertinent to the patient’s needs and then suggest a treatment.202

In Japan and the

United States, AI powered robots are being used to care for the elderly.203

Robots designed for

elderly care are able to provide companionship, even learning the characteristics of the patient.

Robots are also being used to help the elderly bathe, alert medical care incase of an injury, and

encourage physical exercise.204

Political Aspect

In 2013, Christof Heyns, an expert in human rights and international law, wrote a report

for the UN as the Special Rapporteur on Lethal Autonomous Robotics (LARs). He reported that

200

Yan, Jie. (2016). China’s Courts Look to AI for Smarter Judgments Big data and machine

learning to make China’s judicial system more intelligent and efficient. Sixth Tone. 201

Ibid. 202

IBM Watson Oncology. (2016). IBM Watson Oncology Offerings. IBM Watson Health. 203

Broekens, Joost., Heerink, Marcel., Rosendal, Henk. (2009). Assistive Social Robots in

Elderly Care: A Review. Gerontechnology. Vol 8. No 2. Page 94-103. 204

Wada, Kazuyoshi., Shibata, Takanori., Saito, Tomoko., Tanie, Kazuo. (2004). Effects of Ro-

bot-Assisted Activity for Elderly People and Nurses at a Day Service Center. Invited Paper. Pro-

ceeding of the IEEE. Vol. 92. No 11. Page 1780-1788.

50

the Human Rights Council should call on states to stop the testing, development, production and

transfer of LARs.205

He came to this conclusion due to a number of legal reasons. He notes that

states are legally responsible for the actions of LARs but it will be difficult to figure out who in-

side the state should be held responsible for their actions.206

LARs can be hacked or spoofed

which makes legal accountability difficult if it is not clear who was controlling the LARs at the

time of a war crime.207

LARs can break and thus start to target humans indiscriminately.208

Christof Heyns also cites that further developments of AI cannot be foreseen and are likely to

have developmental problems that take time to control or fix.209

Finally, he cautions against the

idea that robots will bring on an age of risk-less war. He expresses that in practice LARs will be

used to not only to attack other robots but people will be targeted too.210

These are difficult

points to politically overcome.

There are some political gains to using LAWS. This is demonstrated by the increase use

of drone strikes from 50 ordered under President Bush to over 400 ordered by President

Obama.211

Unmanned weapons offer U.S. Presidents a low cost way of maintaining a presence

in multiple countries and provide an immediate means of attacking terrorist leaders. An

alternative to unmanned systems would be incredibly expensive or unrealistic.212

205 Heyns, Christof. (2013). Report of the Special Rapporteur on Extrajudicial, Summary or Arbi-

trary Executions. Human Rights Council Twenty-third Session Agenda Item 3. United Nations.

General Assembly. A/HRC/23/47. Page 21. 206 Ibid. Page 14. 207 Ibid. Page 18. 208 Ibid. Page 18. 209 Ibid. Page 18. 210 Ibid. Page 16. 211 Byman, Daniel L. (2013). Why Drones Work: The Case for Washington’s Weapon of Choice.

The Brookings Institute. 212 Ibid.

51

The UN will also be attracted to the utility of peacekeeping LAWS, especially in a less

than lethal configuration. The increased demand for preventive deployments coupled with the

decreased desire from nations to commit troops creates the perfect storm to cause the UN to

obtain and deploy peacekeeping LAWS.213

The UN has struggled with their slow reaction time

to unfolding crisis. The time it takes to motivate countries to commit resources, train and supply

troops, and deploy them takes months, as seen in Rwanda.214

If technology allows the UN to

show it’s presence and support for peaceful resolution without the time and expense of current

peacekeeping, the UN will likely follow that option.

Politically there are also repercussions of soldiers committing war crimes such as torture,

as was done in Abu Ghraib, or disrespectful acts, as done by Marines urinating on dead Taliban

in 2012. A study done by the Surgeon General’s Office in 2006 found that:

Only 47% of Soldiers and only 38% of Marines agreed that non-combatants should be

treated with dignity and respect. Well over a third of Soldiers and Marines reported

torture should be allowed, whether to save the life of a fellow Soldier or Marine (41%

and 44%, respectively) or to obtain important information about insurgents (36% and

39%, respectively).215

The U.S. military is not alone in being accused of war crimes. U.N. Peacekeepers are regularly

accused of rape and sexual assault on those they are sent to protect.216

LAWS could never

commit these crimes. As LAWS become a mature technology it is likely the political advisors

213 United Nations. (2004). UN Report of the High-Level Panel on Threats, Challenges and

Change. "A More Secure World: Our Shared Responsibility.” UN General Assembly. A/59/565.

Page 38. 214 Power, Samantha. (2001).”Bystanders to Genocide: Why the United States Let the Rwandan

Tragedy Happen”. The Atlantic Monthly. 215

Office of The Surgeon General. (2006). Mental Health Advisory Team (MHAT) IV Opera-

tion Iraqi Freedom 05-07 Final Report Office of the Surgeon Multinational Force-Iraq and Office

of The Surgeon General. United States Army Medical Command. Page 35. 216

Defeis, Elizabeth F.. (2008). U.N. Peacekeepers and Sexual Abuse and Exploitation: An End

to Impunity. Washington University Global Studies Law Review.

52

will push for more LAWS on the battlefield to limit the likelihood and opportunity for humans to

commit these crimes.

Military

The U.S. military addresses LAWS from a number of angles. First, there is a DOD

Direction that restricts the development and deployment LAWS. Second, there is a call for

limited use of unmanned systems. This limitation means limited resident knowledge on LAWS

or advocacy for LAWS. Unmanned systems are still in their nascent beginnings and will need

time to become a viable weapon in all domains. Until the hardware is improved, the software of

AI will have limited lethality on the modern battlefield.

As previously pointed out, the US Secretary of Defense signed DOD Directive 3000.09

in 2012 addressing the limitations on autonomous weapons. It establishes, “Autonomous and

semi-autonomous weapon systems shall be designed to allow commanders and operators to

exercise appropriate levels of human judgment over the use of force.”217

There are a number of

exceptions to this policy. First, semi-autonomous weapons can be developed as long as they

require a human to pre-designate targets.218

Second, human supervised autonomous weapons

can be used for defense purposes such as defense of a static base or ship.219

Third, autonomous

weapons may be used if they are non-lethal such as electronic attack or against material targets,

i.e. attacking enemy LAWS.220

However, the DOD directive may soon be obsolete. Ashton

217


Number 3000.09. Deputy Secretary of Defense. 218

Ibid. 219

Ibid. 220

Ibid.

53

Carter, U.S. Secretary of Defense, signed the directive with a sunset clause; it will expire

November 21, 2022.221

In the call for the use of LAWS there are a number of advocates. One such advocate, Dr.

Peter Singer, has called for the U.S. military to invest heavily in unmanned systems. He views

these weapons as the future of warfare and necessary for winning in future combat.222

He asserts

that, “The future of war is robotic because the present of war is robotic.”223

Two examples can

demonstrate the present reality of robotics in war. First, in the Libyan Civil War, a private

contractor provided drone support to the rebel army using off-the-shelf civilian drones.224

Second, in the war on ISIS, every actor is using drones to include Hezbollah, Russia, ISIS, Iraq,

U.S., Iran, Turkey, and Syria.225

The Marine Corps has published its vision for the use of robots in its operating concept

dated September 2016. In it the Commandant reaffirms that the near future of combat will still

be largely human against human but that autonomous robots will also be on the future

battlefield.226

The Marine Corps will integrate RAS with manned systems in what is referred to

as Manned Un-Manned Teaming (MUM-T).227

The Marine Corps RAS are built for ISR, which

then allows the manned systems to carry lethal weapons into combat. The Marine Corps does

employ lethal unmanned systems but they are only semi-autonomous and within the restrictions

221

Ibid. 222

Singer, P.W., August Cole. (2015). The weapons the U.S. needs for a war it doesn’t want.

Reuters. 223


Weapons. YouTube. New America. 31:25. 224

Ibid. 31:36. 225

Ibid. 31:54. 226

The United States Marine Corps. (2016). The Marine Corps Operating Concept How an Ex-

peditionary Force Operates in the 21st Century. Marine Corps Combat Development Command.

Page 6. 227

Ibid. Page 16.

54

set by DOD Directive 3000.09. Other than RAS designed for ISR, the Marine Corps is

developing RAS that are designed for logistical purposes.228

In 2015, the National Defense University (NDU) completed a study on the future of RAS.

In the report, the authors concluded the U.S. military will continue to develop and employ

RAS.229

The report calls for an increase of DOD testing and fielding of RAS, which includes

establishing a national test range specifically for autonomous systems.230

The NDU

acknowledges that the human-machine interfaces need to be improved to increase the trust

between humans and machines.231

The NDU report forcefully states that RAS technology and its

integration into the military force is imperative to U.S. national security. If any potential enemy

develops the technology first then they will have a distinct advantage over the U.S.232

“In ad-

vances AI development as in chess there will be a clear first-mover advantage.”233

Center for a New American Security published multiple articles to argue that LAWS

could be employed ethically. CNAS launched its Ethical Autonomy Project to help countries

understand and overcome the challenges of working with LAWS.234

Paul Scharre, the Director

of the Future of Warfare Initiative at CNAS, writes that the discussion of LAWS needs to be

broken up into three separate subjects: first, the human-machine command and control relation-

228

Ibid. Page 1. 229

Pregmon, Matthew., et al. (2015). Final Report Robotics and Autonomous Systems. The

Dwight D. Eisenhower School for National Security and Resource Strategy. National Defense

University. Fort McNair, Washington D.C. Page 2. 230

Ibid. Page 22. 231

Ibid. Page 21. 232

Ibid. Page 28. 233


Era. Page 11. 234

Scharre, Paul., Horowitz, Michael C. (2015). An Introduction to Autonomy in Weapon Sys-

tems. Center for New American Security. Page 2.

https://www.cnas.org/research/future-of-warfare-initiative

55

ship; second, the complexity of the machine; and third, the type of decision being automated.235

By separating out these discussion points the military can advance LAWS technology in one part

while still adhering to restrictions in another. Paul Scharre also helps clarify levels of control for

the military. He defines human “in the loop” as a human having control over selecting and en-

gaging a target. Next, human “on the loop” refers to a human monitoring the selection and en-

gagement of the targets, intervening when required to do so. Finally, human “out of the loop”

means a human does not need to select or approve the engagement of the target. Additionally,

humans do not monitor the weapon system regularly.236

Advances in Civilian AI that cross over to Military

There are a few civilian base AI robots that mirror the military development, or in some

cases lead the way in development. The Uber project co-funded by Volvo to develop the wide

use of autonomous cars, has a similar goal to the unmanned logistical vehicles spoken of by the

Marine Corps Operating Concept. Watson technology can be adapted to design military plans or

suggest new ideas to commanders facing the complex battlefields of the future. As these

technologies proliferate through civil society they will likely be adapted by the military. Since

these systems are not inherently lethal, they would not be prohibited under the possible Protocol

VI.

Technological Limitation

235

Ibid. Page 5-6. 236

Ibid. Page 8.

56

There are technological limitations on AI and thus LAWS. The technology that is lob-

bied for or against currently does not exist. There are disputes as to when singularity will take

place. Experts have wide timeline for when this will take place. OpenCog advertises AI will be

available by 2040.237

Whereas Vernor Vinge believes that singularity will happen sometime

around 2030.238

Even though the timeline is not clear, it is clear that there will be a progression

towards singularity then ASI. As technology progresses, weapons developers will need to ensure

that the autonomy in new weapon systems fall within the guidelines of IHL and state enforced

rules of engagement.239

Differences in Costs Between Humans and Robots

Comparing the total costs of humans to robots is difficult to capture. There are a few

general trends that help shed light on the subject. In 2012, the Pentagon comptroller, Under Sec-

retary Robert Hale, stated that the average soldier in Afghanistan costs $850,000 per year.240

In

2012, the Military Retirement Fund pays current beneficiaries a total of $51.7 billion a year.241

This expense would reduce for any military force that replaces humans with robots. In 2002, the

price to recruit, train, and equip a Marine was $44,887.242

This cost represents the price point for

procuring LAWS that would directly replace humans in the military force.

237 Pitt, Joel. (2010). Vision. OpenCog.org. 238 Wolens, Doug. (2009). Singularity 101 with Vernor Vinge. Humanity Plus Magazine. 239 Quintana, Elizabeth. (2012). The Ethics and Legal Implications of Military Unmanned Vehi-

cles Head of Military Technology & Information Studies Royal United Services Institute for De-

fence and Security Studies. Page 15. 240

Shaughnessy, Larry. (2012). One soldier, one year: $850,000 and rising. CNN. 241

Congress of The United States Congressional Budget Office. (2012). Costs of Military Pay

and Benefits in the Defense Budget. Page 27. 242

Olick, Diana. (2002). An army of one carries a high price. NBC News.

http://security.blogs.cnn.com/2012/02/28/one-soldier-one-year-850000-and-rising/

57

Summary

There are a number of organizations working to bring about LAWS while others try to

ban them. There are a number of NGOs that are working through legislation to ban LAWS. If

they are successful, it will reduce the chances of deaths caused by LAWS and their unproved /

unreliable technology. Politicians will be apprehensive to use machines to decide who lives and

who dies in a conflict. A LAWS breaking in a conflict zone and killing civilians

indiscriminately will be difficult for any politician to explain. On the other hand, politicians will

be pressured to use more unmanned systems including LAWS to reduce occurrences of human

soldier raping or assaulting the citizenry. Militaries are bureaucracies and thus will be slow to

advocate for then utilize LAWS. Once their potential enemies have LAWS then militaries will

quickly advocate for them. The final factor as to whether or not LAWS is utilized is the pace of

technology. Whether the weapons are wanted or feared, they don’t exist yet. Until they do exist

it is unclear which forces will press harder for their convictions and which will turn silent,

acknowledging failure to realize their vision.

58

CHAPTER 6

CONCULSION

Potential Influences of LAWS

As LAWS technology develops and evolves, there are a number of potential influencers

on international decision makers and foreign policy. As automation proliferates, there will be

less apprehension towards the use of automation in the military. If LAWS are found to be quan-

tifiably less likely to make a mistake in conflict then the discourse on autonomous weapons may

change from the use of robots being unethical to being ethical. This would be a polar shift in in-

ternational pressure within the UN from banning robots to potentially banning humans from con-

flict.

The national security perspective about LAWS will change as it becomes reality. Chang-

es to opinions will take time to come to fruition. If military industry can bring down the price

point of LAWS to a level below the cost of military personnel then there would be a military and

congressional pressure to lower operating costs of national defense security. If LAWS can prove

to be better at fighting than military personnel there is likely to be a demand signal for more

LAWS and less humans by the military services. As these thresholds are achieved, the national

security advisors will be more likely to encourage the use of LAWS in a perspective conflict.

Robots with weapons are already a reality on the modern battlefield. More autonomy is

coming to the military. National security advisors and military commanders will have an in-

creasing number of options to prosecute a target or military objective without exposing military

personnel to danger. The level of autonomy that is accepted in the application of lethal force still

has years to be decided and will likely change as technology matures.

59

Further, if LAWS replace humans on the battlefield it will open up new options for deci-

sion-makers. If there is no risk to human lives for their own force, decision-makers may more

incline to use military force than when there was risk to U.S. personnel. Politically it may be

easier to dedicate U.S. forces knowing that there will not be any images of wounded or kicked

service members in the news. This thought process could lead to congressional authorization for

use of force that only includes the use of LAWS. The use of a LAWS-only military action could

further increase if the enemy combatant also has LAWS units. Essentially, this would mean that

military hardware could be used to attack only military hardware. In practice, it is likely that

humans would still be killed or wounded but at a reduced rate.

Authorization of a LAWS-only military force could also increase the number of UN au-

thorization of force. If the member states of the UN believe that LAWS are reliable enough to

only target combatants then they may be more likely to authorize the use of force. Additionally,

the member states may be more likely to provide forces in the form of all LAWS units.

Summary

As LAWS turn from science fiction to science fact, the capabilities and limitations will

help inform the conversation. The current discourse on LAWS can be organized into the two

sides of the argument, for or against LAWS. One side of the controversy views LAWS as

dangerous and should be banned. LAWS maybe dangerous for several reasons, first, the

artificial intelligence in the LAWS will be smarter than the humans leading them. Because they

are learning machines, it will be impossible to know what decisions the LAWS will make. These

decisions could be as significant as killing the wrong person based on poor data inputs. Also, a

decision could be made to change its ethical restraints and put a lower value on human life than

60

is allowed. Second, the LAWS could break, leading it to lose management of its fire controls

and potentially to start killing indiscriminately, combatants and non-combatants alike. This leads

to a difficult legal position of who can be held responsible. The list of responsible parties

includes the operator, the commander, the manufacturer, the designer, the state government, or

even a hacker that was not known at the time. A list this long leads one to believe that no one

will be held accountable for any wrongful actions taken by a LAWS.

The discourse for the other side argues that pandora’s box has already been opened. It

doesn’t matter if LAWS are dangerous, they are going to be a part of future conflict. The real

danger lies in allowing a potential enemy to obtain them first. Those that argue for LAWS be-

lieve that it is likely that restrictions will be instituted early in their development followed by a

slow attenuation of those restrictions. Early restrictions will require a human in the loop, then

humans to supervise the LAWS, and finally a loose supervision of the machines.243

The support-

ing discourse argues for the possibility of providing ethical restraints to the unmanned systems.

If those restraints are broken, the potential damage caused by LAWS will be limited enough that

it will still be less frequent than humans committing crimes in war. LAWS will have limitations

on their reasoning power but humans are also prone to errors.244

Advocates for LAWS also

assert that autonomous weapons are needed to counter the enemy LAWS as well as to limit the

loss of human life.

As technology brings LAWS closer to reality the discourse on them will be refined and

clarified. The true dangers and benefits of this burgeoning technology will be revealed. Because

243


Lubell TEDxUniversityofEssex. YouTube.com. 5:53. 244

Scharre, Paul. (2016). Autonomous Weapons and Operational Risk. Ethical Autonomy Pro-

ject Senior Fellow and Director of the 20YY Future of Warfare Initiative. Center for a New

American Security. Page 5

61

full autonomy powered by ASI is not likely until 2030 or 2040 the discourse will continue

through that time and beyond until ASI is weaponized.

Suggested Follow-on Research

There are still many more questions to be answered about LAWS and the repercussions

of their employment. This includes questions held by human rights groups which are largely

against LAWS. Politicians still need further research done to help clarify the political ramifica-

tions of deploying LAWS. Additionally, militaries need to study the integration of LAWS into

their doctrine which will necessarily change military options that can be offered to civilian lead-

ers.

In the near future, human rights organizations will continue to work on legislation to ban

the use of LAWS. Concurrently, these same organizations will want to encourage the develop-

ment of robotics and AI that can be helpful for humanitarian aid. For example, Zipline, a startup

company that delivers medical supplies to hard to reach areas in Rwanda for cheaper and faster

than a ground currier, is acclaimed for its improvements in humanitarian service.245

AI could

benefit humanitarian aid efforts in the same way Watson is being used to benefit the medical in-

dustry. James Barrat explains, “Every one of these [computer scientists] was convinced that in

the future all the important decisions governing human lives will be made by machines… Many

think this will take place within their lifetimes.”246

The question now remains: How do you de-

velop this technology for humanitarian work and ensure that it is not later weaponized?

245

Zipline. (2016). The Future of Healthcare is Out for Delivery. Zipline Homepage. 246


Era. Page 3.

62

Political administrations still require research into what technological thresholds should

be achieved to make employment of LAWS more politically acceptable than deployments of

human soldiers. This research will likely be broken down by domain to include undersea, sur-

face ships, UGVs, drones, and cyber.

The militaries still need to conceptualize how LAWS will be organized into current

manned units. Will LAWS replace or augment manned units and to what extent? This will drive

the military options offered to the politicians, for instance, if the military could deploy an un-

manned unit then the opportunity of fighting a conflict without the loss of their own soldiers

would be appealing. This line of reasoning assumes that the military will figure out how to en-

sure that there are clear, ethical boundaries for LAWS.

A comprehensive study examining the price differences between a robot and human

would enrich the debate about the merits of both options. Military personnel have recruitment,

training, pay, allowances, housing, and medical costs which accrue on an annual basis. Once a

soldier leaves the military service, there is a potential expense of disability and/or retirement pay.

If the soldier’s life is lost, there are burial, insurance, and allowance payments due, which can

total in excess of $500,000. The only costs that transfer over would be initial training (produc-

tion), feeding (gas or electric power), and medical (maintenance). Robots also have high ex-

penditures in research and development that do not apply to a human soldier. Conducting a

comprehensive study in the costs would help in understanding what an appropriate price point

would be for LAWS that directly replace a human on the battlefield.

Final Thoughts

63

It is difficult to conceptualize how dangerous a weapon will be when it does not exist.

Eliezer Yudkowsky affirms, “Artificial Intelligence is not settled science.”247

The vision of what

LAWS will be capable of doing is particularly hard to visualize because they will incorporate

several different technologies that are still being developed. At the heart of the issue is AI. Will

AI start to develop new AI that operates outside the ethical constraints of the human built AI?

Vernor Vinge states, “I think that in the relatively near historical future, humans, using

technology, will be able to create, or become, creatures of superhuman intelligence.”248

There is also a certain level of anxiety about arming AI and allowing it to fight

unsupervised in combat. Essentially every new weapon on the battlefield brings a similar

anxiety with it. Two examples are chemical weapons and mines. Chemical weapons have been

banned and have a high threshold of weaponizing them, thus there are limited examples of them

being used in conflict. Mines have been ban also but have a low threshold of obtaining and thus

are used in modern conflict, particularly by non-state actors. Because LAWS will have a low

threshold of being obtained, it is likely that a ban will have limited effect on LAWS in conflict.

Countries and NGOs will need to develop technology to counter LAWS in conflict similar to the

current arrangement done to counter mines.

247

Yudkowsky, Eliezer. (2008). Artificial Intelligence as a Positive and Negative Factor in

Global Risk. Page 1. 248

Wolens, Doug. (2009). Singularity 101 with Vernor Vinge. Humanity Plus Magazine.

64

REFERENCES

Ackerman, Evan. (2012). Latest AlphaDog Robot Prototypes Get Less Noisy, More Brainy.

IEEE Spectrum. http://spectrum.ieee.org/automaton/robotics/military-robots/latest-ls3-alphadog-

prototypes-get-less-noisy-more-brainy. Accessed Nov 2016.

Amazon Prime Air. (2016). Amazon Prime Air. Website. Amazon.

https://www.amazon.com/b?node=8037720011. Accessed Nov 2016.



School of Law. YouTube. https://www.youtube.com/watch?v=b5mz7Y2FmU4. Accessed Oct

2016.

Anderson, Kenneth., Daniel Reisner., Matthew Waxman. (2014). Adapting the Law of Armed

Conflict to Autonomous Weapon Systems. Stockton Center for the Study of International Law.

U.S. Naval War College. https://www.usnwc.edu/getattachment/a2ce46e7-1c81-4956-a2f3-

c8190837afa4/dapting-the-Law-of-Armed-Conflict-to-Autonomous-We.aspx. Accessed Nov

2016.

Anderson, Susan Leigh. (2005). Asimov’s “Three Laws of Robotics” and Machine Metaethics.

University of Connecticut. Dept. of Philosophy, 1 University Place, Stamford, CT 06901.

http://www.aaai.org/Papers/Symposia/Fall/2005/FS-05-06/FS05-06-002.pdf. Accessed Nov

2016.

Article 36. (2013). Killer Robots: UK Government Policy of Fully Autonomous Weapons.

http://www.article36.org/wp-content/uploads/2013/04/Policy_Paper1.pdf. Accessed Nov 2016.

ASIMO. (2016). ASIMO The World’s Most Advanced Humanoid Robot History Website.

http://asimo.honda.com/asimo-history/. Accessed Nov 2016.

Asimov, Isaac. (2004). I, Robot (The Robot Series Book 1). Random House Publishing Group.

Kindle Edition.

Association for the Advancement of Artificial Intelligence. (2016). Association for the Ad-

vancement of Artificial Intelligence Homepage. http://www.aaai.org/home.html. Accessed Nov

2016.

Automated Insights. (2016). Automated Insights Company Webpage. Automated Insights.

https://automatedinsights.com/company. Accessed Nov 2016.

BAE Systems. (2016). Taranis. BAE Systems. http://www.baesystems.com/en/product/taranis.

Accessed Nov 2016.


Era. Thomas Dunne Books. New York, New York.

65

Beckhusen, Robert. (2015). Get Ready, NATO: Could Russia Build Lethal Combat Robots?. Na-

tional Interest. http://nationalinterest.org/blog/the-buzz/get-ready-nato-could-russia-build-lethal-

combat-robots-13954. Accessed Nov 2016.

Bergen, Peter., Singer, Peter. (2015). Machines That Kill: Will We Rely on Autonomous Weap-

ons. YouTube. New America. https://www.youtube.com/watch?v=g2P1KaQ4AC4. Accessed

Nov 2016.

Boston Dynamics. (2016). LS3 - Legged Squad Support Systems. Boston Dynamics. Webpage.

http://www.bostondynamics.com/robot_ls3.html. Accessed Nov 2016.

Bostrom, Nick., Yudkowsky, Eliezer. (2014). The Ethics of Artificial Intelligence. Machine In-

telligence Research Institute. https://intelligence.org/files/EthicsofAI.pdf. Accessed October

2016.

Bostrom, Nick. (2003). Ethical Issues in Advanced Artificial Intelligence. Oxford University.

United Kingdom. http://www.nickbostrom.com/ethics/ai.html#_ftn7. Accessed November 2016.

Bostrom, Nick (2008). How Long Before Superintelligence?. Oxford Future of Humanity Insti-

tute. Faculty of Philosophy & Oxford Martin School. University of Oxford.

http://www.nickbostrom.com/superintelligence.html. Accessed Nov 2016.

Bostrom, N. (2016): “Strategic Implications of Openness in AI Development”, Technical Report

#20161. Future of Humanity Institute. Oxford University. pp. 1-26.

http://www.fhi.ox.ac.uk/reports/2016-1.pdf. Accessed Nov 2016.

Bostrom, Nick (2014). Superintelligence: Paths, Dangers, Strategies. OUP Oxford. Kindle

Edition.

Bowcott, Owen. (2015). UK opposes international ban on developing 'killer robots’. The

Guardian. https://www.theguardian.com/politics/2015/apr/13/uk-opposes-international-ban-on-

developing-killer-robots. Accessed Nov 2016.

British Broadcasting Corporation. (2014). Computer AI passes Turing test in 'world first’. BBC

Technology. http://www.bbc.com/news/technology-27762088. Accessed Nov 2016.

Broekens, Joost., Heerink, Marcel., Rosendal, Henk. (2009). Assistive Social Robots in Elderly

Care: A Review. Gerontechnology. Vol 8. No 2. Page 94-103.

Burrus, Daniel. (2014). The Internet Of Things Is Far Bigger Than Anyone Realizes. Burrus

Research. Wired Magazine. https://www.wired.com/insights/2014/11/the-internet-of-things-

bigger/. Accessed Nov 2016.

66

Byman, Daniel L. (2013). Why Drones Work: The Case for Washington’s Weapon of Choice.

The Brookings Institute. https://www.brookings.edu/articles/why-drones-work-the-case-for-

washingtons-weapon-of-choice/. Accessed Nov 2016.

Campaign to Stop Killer Robots. (2016). Ban support grows, process goes slow. Campaign to

Stop Killer Robots. https://www.stopkillerrobots.org/2016/04/thirdmtg/. Accessed Nov 2016.

Campaign to Stop Killer Robots. (2016). Campaign to Stop Killer Robots Home Page. Website.

http://www.stopkillerrobots.org. Accessed Nov 2016.

Campaign to Stop Killer Robots. (2016). Call to Action. Website. Campaign to Stop Killer Ro-

bots. http://www.stopkillerrobots.org/call-to-action/ Accessed Nov 2016.

Campaign to Stop Killer Robots. (2016). Third UN meeting opens April 11. Website. Campaign

to Stop Killer Robots. https://www.stopkillerrobots.org/2016/04/thirdccw/. Accessed Nov 2016.


Number 3000.09. Deputy Secretary of Defense.

http://www.dtic.mil/whs/directives/corres/pdf/300009p.pdf. Accessed Nov 2016.

Cauer, Emil., Wolfgang Mathis, Rainer Pauli (2000). Life and Work of Wilhelm Cauer (1900 –

1945). Perpignan, France.

https://www.cs.princeton.edu/courses/archive/fall03/cs323/links/cauer.pdf. Accessed Nov 2016.

Cockburn, Andrew. (2015). Kill Chain: The Rise of the High-Tech Assassins. Henry Holt and

Co. New York, New York.

Congress Of The United States Congressional Budget Office. (2012). Costs of Military Pay and

Benefits in the Defense Budget. https://www.cbo.gov/sites/default/files/112th-congress-2011-

2012/reports/11-14-12-MilitaryComp_0.pdf. Accessed Dec 2016.

DeepMind. (2016). DeepMind About Website. DeepMind. https://deepmind.com/about/. Ac-

cessed Nov 2016.

Defeis, Elizabeth F. (2008). U.N. Peacekeepers and Sexual Abuse and Exploitation: An End to

Impunity. Washington University Global Studies Law Review. Volume 7. Issue 2. Page 184-

214.

Defense Advanced Research Projects Agency. (2013). DARPA Envisions the Future of Machine

Learning. Automated tools aim to make it easier to teach a computer than to program it. News

And Events. DARPA Envisions the Future of Machine Learning. http://www.darpa.mil/news-

events/2013-03-19a. Accessed Nov 2016.

Department of the Navy. (2004). The Navy Unmanned Undersea Vehicle (UUV) Master Plan.

The Department of the Navy. United States of America.

http://www.navy.mil/navydata/technology/uuvmp.pdf. Accessed Nov 2016.

67

Dwoskin, Elizabeth., Evelyn M. Rusli. (2015).The Technology that Unmasks Your Hidden Emo-

tions Using Psychology and Data Mining to Discern Emotions as People Shop, Watch Ads;

Breeding Privacy Concerns. Tech. The Wall Street Journal.

http://www.wsj.com/articles/startups-see-your-face-unmask-your-emotions-1422472398. Ac-

cessed Nov 2016.

Dyer, John. (2016). Ivan the Terminator: Russia Is Showing Off Its New Robot Soldier. Vice

News. https://news.vice.com/article/ivan-the-terminator-russia-is-showing-off-its-new-robot-

soldier. Accessed Nov 2016.

Exponent. (2016). MARCbot. Exponent Webpage.

http://www.exponent.com/experience/marcbot/?pageSize=NaN&pageNum=0&loadAllByPageSi

ze=true. Accessed Nov 2016.

Fei-Fei, Li. (2015). Fei-Fei Li: How we're teaching computers to understand pictures. Ted Talks.

https://www.ted.com/talks/fei_fei_li_how_we_re_teaching_computers_to_understand_pictures?l

anguage=en#t-1066099. Accessed Nov 2016.

Fei-Yue, Wang., Shuming Tang. (2004). Artificial societies for integrated and sustainable devel-

opment of metropolitan systems. IEEE Intelligent Systems Volume: 19. Issue:

4.http://ieeexplore.ieee.org/document/1333039/?part=1. Accessed Nov 2016.

Future of Life Institute. (2016). Autonomous Weapons: An Open Letter From Ai & Robotics

Researchers. http://futureoflife.org/open-letter-autonomous-weapons/. Accessed October 2016.


ments in Artificial Intelligence on Society. http://futureoflife.org/2016/05/06/computers-gone-

wild/. Accessed Nov 2016.

Future of Humanity Institute. (2016). Future of Humanity Institute Team Website. University of

Oxford. https://www.fhi.ox.ac.uk/about/the-team/. Accessed Nov 2016.

Future of Humanity Institute. (2016). Future of Humanity Institute About Website. University of

Oxford. https://www.fhi.ox.ac.uk/about/mission/. Accessed Nov 2016.


Pages 14-16. New York, NY, USA. http://delivery.acm.org/10.1145/2140000/2133812/p14-

gel-

ler.pdf?ip=97.83.201.44&id=2133812&acc=OPEN&key=4D4702B0C3E38B35%2E4D4702B0

C3E38B35%2E4D4702B0C3E38B35%2E6D218144511F3437&CFID=862577757&CFTOKEN

=14543452&__acm__=1478785082_35ec21690adc0cb005348437ef750e05. Accessed Nov

2016.

Ghose, Tia. (2015). 3D Computer Chips Could Be 1,000 Times Faster Than Existing Ones. Live

Science. http://www.livescience.com/52207-faster-3d-computer-chip.html. Accessed Nov 2016.

68

Good, Irving John. (1965). “Speculations Concerning the First Ultraintelligent Machine.” In Ad-

vances in Computers, ed. by Franz L. Alt and Morris Rubinoff, 31–88. Vol. 6. New York:

Academic Press. http://deeprecursion.com/file/2012/05/9292221-Good65ultraintelligent.pdf. Ac-

cessed Nov 2016.

Google Self-Driving Car. (2016). Google Self-Driving Car Website.

https://www.google.com/selfdrivingcar/. Accessed Nov 2016.

Goose, Steve. (2016). Statement by Human Rights Watch to the Preparatory Committee for the

Fifth Review Conference of the Convention on Conventional Weapons.

https://www.hrw.org/news/2016/08/31/statement-human-rights-watch-preparatory-committee-

fifth-review-conference. Accessed Nov 2016.

Grauman, Kristen., Bastian Leibe. (2011). Visual Object Recognition. Synthesis Lectures On

Computer Vision # 1. http://cs.gmu.edu/~kosecka/cs482/grauman-recognition-draft-27-01-

11.pdf. Accessed Nov 2016.

Greenemeier, Larry. (2009). Computers have a lot to learn from the human brain, engineers say.

Scientific America. https://blogs.scientificamerican.com/news-blog/computers-have-a-lot-to-

learn-from-2009-03-10/. Accessed Nov 2016.

Gubrud, Mark. (2014). Is Russia Leading the World to Autonomous Weapons?. International

Committee For Robot Arms Control. http://icrac.net/2014/05/is-russia-leading-the-world-to-

autonomous-weapons/. Accessed Nov 2016.

Hambling, David. (2014). Russia Wants Autonomous Fighting Robots, and Lots of Them

Putin's military is busy building autonomous tanks with mounted machine guns and unmanned,

amphibious Jeep-size vehicles. Popular Mechanics.

http://www.popularmechanics.com/military/a10511/russia-wants-autonomous-fighting-robots-

and-lots-of-them-16787165/. Accessed Nov 2016.

Hanson Robotics (2016). Hanson Robotics Company Overview. Hanson Robotics Limited.

http://www.hansonrobotics.com/wp-content/uploads/2016/09/Hanson-Robotics-Overview.pdf.

Accessed Nov 2016.

Heyns, Christof. (2013). Report of the Special Rapporteur on Extrajudicial, Summary or Arbi-

trary Executions. Human Rights Council Twenty-third Session Agenda Item 3. United Nations.

General Assembly. A/HRC/23/47.

Human Rights Watch., International Human Rights Clinic. (2016). Killer Robots and the Con-

cept of Meaningful Human Control Memorandum to Convention on Conventional Weapons

(CCW) Delegates.

https://www.hrw.org/sites/default/files/supporting_resources/robots_meaningful_human_control

_final.pdf. Accessed Nov 2016.

69

Human Rights Watch. (2016). Statement to the Convention on Conventional Weapons Fifth Re-

view Conference Preparatory Meeting on Main Committee II. Delivered by Mary Wareham. Ad-

vocacy Director at Human Rights Watch and Global Coordinator for the Campaign to Stop Killer

Robots. https://www.hrw.org/news/2016/09/02/statement-convention-conventional-weapons-

fifth-review-conference-preparatory. accessed October 2016.

IBM Research. (2016). Cognitive Computing. IBM Research Website.

http://www.research.ibm.com/cognitive-computing/index.shtml#fbid=vmpqdEURf04. Accessed

Nov 2016.

IBM Watson Health. (2016). IBM Watson Health Website. https://www.ibm.com/watson/health/.

Accessed Nov 2016.

IBM Watson Oncology. (2016). IBM Watson Oncology Offerings. IBM Watson Health.

http://www.ibm.com/watson/health/oncology/. Accessed Nov 2016.

IBM Watson Products. (2016). IBM Watson Products Website.

http://www.ibm.com/watson/products.html. Accessed Nov 2016.

International Committee of the Red Cross. (2014). Expert Meeting, Autonomous Weapon Sys-

tems Technical, Military, Legal, and Humanitarian Aspects Report. International Committee of

the Red Cross. Geneva, Switzerland.

IEEE. (2016). 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO).

http://www.ieee.org/conferences_events/conferences/conferencedetails/index.html?Conf_ID=39

756&WT.mc_id=uem_robio_16. Accessed Nov 2016.

Israel Aerospace Industries. (2016). Harp Loitering Weapon System. Israel Aerospace Industries.

Missile Division. http://www.iai.co.il/Sip_Storage//FILES/6/41656.pdf. Accessed Nov 2016.

Jeffries, Adrianne. (2014). Only five countries actually want to ban killer robots, Cuba, Pakistan,

Egypt, Ecuador, and the Vatican backed a lethal autonomous weapons ban at the UN; everyone

else wasn't sure. The Verge. http://www.theverge.com/2014/5/16/5724538/what-happened-at-

the-un-killer-robot-debate. Accessed Nov 2016.

Joint Chiefs of Staff. (2016). Joint Operating Environment 2035, The Joint Force in a Contested

and Disordered World. http://www.dtic.mil/doctrine/concepts/joe/joe_2035_july16.pdf. Ac-

cessed Nov 0216.

Kongsberg. (2016). Naval Strike Missile. Website. Kongsberg Defence Systems. Kongsberg

Gruppen. http://www.kongsberg.com/en/kds/products/missilesystems/navalstrikemissile/. Ac-

cessed Nov 2016.

Kumagai, J. (2007). A Robotic Sentry For Korea's Demilitarized Zone. Browse Journals & Mag-

azines. IEEE Spectrum. Volume: 44 Issue: 3.

http://ieeexplore.ieee.org/document/4119213/?part=1. Accessed Nov 2016.

70

Kurzweil, Ray. (2001). The Law of Accelerating Returns. Kurzweil AI Website.

http://www.kurzweilai.net/the-law-of-accelerating-returns. Accessed Nov 2016.

Kurzweil, Ray. (2005). The Singularity Is Near: When Humans Transcend Biology. Penguin

Publishing Group. Kindle Edition.

Littlefield, Scott. (2016). Anti-Submarine Warfare (ASW) Continuous Trail Unmanned Vessel

(ACTUV) Anti-Submarine Warfare (ASW) Continuous Trail Unmanned Vessel (ACTUV). De-

fense Advanced Research Projects Agency. http://www.darpa.mil/program/anti-submarine-

warfare-continuous-trail-unmanned-vessel. Accessed Nov 2016.

Lockheed Martin Corporation. (2016). Autonomous Mobility Appliqué System (AMAS).

Lockheed Martin Corporation. Website. http://lockheedmartin.com/us/products/amas1.html.

Accessed Nov 2016.


Lubell TEDxUniversityofEssex. YouTube.com.

https://www.youtube.com/watch?v=x8_7XTDnAEk. Accessed Nov 2016.

Machine Intelligence Research Institute. (2016). Research Webpage.

https://intelligence.org/research/. Accessed Nov 2016.

Marshall, Aarian., Alex Davies. (2016). How Pittsburgh Birthed The Age Of The Self-Driving

Car. Transportation. Wired Magazine. https://www.wired.com/2016/08/pittsburgh-birthed-age-

self-driving-car/. Accessed Nov 2016.

Massachusetts Institute of Technology. (2016). CSAIL Mission. Webpage. Computer Science

and Artificial Intelligence Laboratory. https://www.csail.mit.edu/about/CSAIL_Mission. Ac-

cessed Nov 2016.

Massachusetts Institute of Technology. (2016). Speech-Based Interface For Medical Data Ac-

cess. Webpage. Computer Science and Artificial Intelligence Laboratory.

https://www.csail.mit.edu/node/1965. Accessed Nov 2016.

Massachusetts Institute of Technology. (2016). Spoken Dialogue System For English Language

Learning. Webpage. Computer Science and Artificial Intelligence Laboratory.


Massachusetts Institute of Technology. (2016). Vision-Based Automated Behavior Recognition.

Webpage. Computer Science and Artificial Intelligence Laboratory.


MBDA Systems. (2016). MBDA Demonstrates Brimstone Missile Live Firing from

Apache Helicopter. MBDA Missile Systems. http://www.mbda-systems.com/air-

dominance/brimstone/. Accessed Nov 2016.

71

Mobile Eye. (2016). Company Overview. Mobile Eye Website.

http://www.mobileye.com/about/. Accessed Nov 2016.

Moore, Gordon. (2016). Moore’s Law Home Page. http://www.mooreslaw.org. Accessed Nov

2016.

Nass, Clifford., Scott Brave. (2005). Wired for Speech: How Voice Activates and Advances the

Human-Computer Relationship. Kindle Edition. MIT Press. Cambridge Massachusetts.

Natural Language Processing Group. (2016). Research Software Engineer - Natural Language

Processing Group Website. Stanford University.

https://symsys.stanford.edu/viewing/textadvertisement/12413. Accessed Nov 2016.

Neild, Barry. (2016). Why your next hotel will be staffed by robots. CNN.

http://www.cnn.com/2016/03/17/hotels/hotels-robot-future-travel/. Accessed Nov 2016.

Newnham, Danielle. (2015). The Story Behind Siri And the man who made her. The Startup.

https://medium.com/swlh/the-story-behind-siri-fbeb109938b0#.8016kyncu. Accessed Nov 2016.

Northrop Grumman. (2016). X-47B UCAS Makes Aviation History…Again!. Webpage.

Northrop Grumman.

http://www.northropgrumman.com/Capabilities/X47BUCAS/Pages/default.aspx. Accessed Nov

2016.

Office of The Surgeon General. (2006). Mental Health Advisory Team (MHAT) IV Operation

Iraqi Freedom 05-07 Final Report Office of the Surgeon Multinational Force-Iraq and Office of

The Surgeon General. United States Army Medical Command.

https://archive.org/stream/551721-mental-health-advisory-team-mhat-iv/551721-mental-health-

advisory-team-mhat-iv_djvu.txt. Accessed Nov 2016.

Olick, Diana. (2002). An army of one carries a high price. NBC News.

http://www.nbcnews.com/id/3072945/t/army-one-carries-high-price/#.WEMTFza7p7g. Ac-

cessed Dec 2016.

Omohundro, Stephen. (2007). Stanford Colloquium: Self-Improving Artificial Intelligence.

Youtube.com. https://www.youtube.com/watch?v=CwXQKk_lmzQ. Accessed Nov 2016.

Ord, Toby. (2014). Drones, counterfactuals, and equilibria:Challenges in evaluating new military

technologies. Future of Humanity Institute. https://www.fhi.ox.ac.uk/wp-

content/uploads/Drones-counterfactuals-and-equilibria-Challenges-in-evaluating-new-military-

technologies.pdf. Accessed Nov 2016.

72

Oxford Brookes University. (2016). Department of Computing and Communications Technolo-

gies. Oxford Brookes University. http://cct.brookes.ac.uk/research/isec/computer-

vision/projects/index.html. Accessed Nov 2016.

Oxford Brookes University. (2016). Tensor modeling for identity recognition from gait and ac-

tivity recognition. Department of Computing and Communications Technologies. Oxford

Brookes University. http://cct.brookes.ac.uk/research/isec/artificial-

intelligence/themes/computer-vision/gait-id.html. Accessed Nov 2016.

Patt, Dr. Daniel. (2016). Aircrew Labor In-Cockpit Automation System (ALIAS). Aircrew Labor

In-Cockpit Automation System. Program Information. Defense Advanced Research Projects

Agency. http://www.darpa.mil/program/aircrew-labor-in-cockpit-automation-system. Accessed

Nov 2016.

Pawlyk, Oriana. (2016). Firm Plans to Build Autonomous Huey Helicopters. Defense Tech.

http://www.defensetech.org/2016/11/11/autonomous-huey-

helicopter/?comp%3D1199442010954%26rank%3D0. Accessed Nov 2016.

Prengaman, Richard J., Edward C. Wetzlar., Robert J. Bailey. (2001). Integrated Ship Defense

Johns Hopkins Apl Technical Digest. Volume 22. Number 4.

http://www.jhuapl.edu/techdigest/TD/td2204/Prengaman.pdf. Accessed Nov 2016.

Pitt, Joel. (2010). Vision. OpenCog.org. http://opencog.org/vision/. Accessed Nov 2016.

Power, Samantha. (2001).”Bystanders to Genocide: Why the United States Let the Rwandan

Tragedy Happen”. The Atlantic Monthly.

http://www.theatlantic.com/magazine/archive/2001/09/bystanders-to-

genocide/304571/?single_page=true%22. Accessed Nov 2016.

Pregmon, Matthew., et al. (2015). Final Report Robotics and Autonomous Systems. The Dwight

D. Eisenhower School for National Security and Resource Strategy. National Defense Universi-

ty. Fort McNair, Washington D.C. http://es.ndu.edu/Portals/75/Documents/industry-

study/reports/2015/es-is-report-robotics-autonomous-systems-2015.pdf. Accessed Nov 2016.

Pyer, Douglas. (2013). The Rise of the Machines, Why Increasingly “Perfect” Weapons Help

Perpetuate our Wars and Endanger Our Nation. Military Review. March-April 2013.

http://usacac.army.mil/CAC2/MilitaryReview/Archives/English/MilitaryReview_20130430_art0

05.pdf. Accessed Nov 2016.

Quintana, Elizabeth. (2012). The Ethics and Legal Implications of Military Unmanned Vehicles

Head of Military Technology & Information Studies Royal United Services Institute for Defence

and Security Studies. http://www.alfredoroma.it/wp-content/uploads/2012/05/RUSI_ethics.pdf.

Accessed Nov 2016.

Raytheon. (2016). Phalanx Close-In Weapon System Last Line of Defense for air, land and sea.

Website. http://www.raytheon.com/capabilities/products/phalanx/. Accessed Nov 2016.

73

Rheinmetall Defence. (2016). Air Defence Systems Guarding against the threat from above.

Rheinmetall Defence. Air Defence Systems. http://www.rheinmetall-

defence.com/en/rheinmetall_defence/systems_and_products/air_defence_systems/index.php.

Accessed Nov 2016.

The Robotics Institute. (2015). Uber, Carnegie Mellon Announce Strategic Partnership. School

of Computer Science. Carnegie Mellon University.

http://www.ri.cmu.edu/news_view.html?news_id=395&menu_id=239. Accessed Nov 2016.

Rowan, David. (2015). DeepMind: inside Google's super-brain. Wired Magazine.

http://www.wired.co.uk/article/deepmind. Accessed Nov 2016.

RT Staff. (2015). Japan to Create More User-Friendly Elderly Care Robots The Japanese gov-

ernment said some of the elderly care robots built have been too large and too expensive.

http://www.roboticstrends.com/article/japan_to_create_more_user_friendly_elderly_care_robots/

medical. Accessed Nov 2016.

Russakovsky, Olga., Jia Deng., Hao Su., Jonathan Krause., Sanjeev Satheesh., Sean Ma.,

Zhiheng Huang., Andrej Karpathy., Aditya Khosla., Michael Bernstein., Alexander C. Berg., Li

Fei-Fei.(2015). ImageNet Large Scale Visual Recognition Challenge. IJCV, 2015 paper.

Salmon, Felix., Jon Stokes. (2010). Algorithms Take Control of Wall Street. Wired Magazine.

https://www.wired.com/2010/12/ff_ai_flashtrading/. Accessed Nov 2016.

Sassoli, Marco., et al. (2014). Autonomous Weapons Systems Under International Law. Geneva

Academy of International Humanitarian Law and Human Rights. YouTube.

https://www.youtube.com/watch?v=Clw2KxBu03c. Accessed Nov 2016.

Scharre, Paul. (2016). Autonomous Weapons and Operational Risk. Ethical Autonomy Project


can Security. https://s3.amazonaws.com/files.cnas.org/documents/CNAS_Autonomous-

weapons-operational-risk.pdf. Accessed Nov 2016.

Scharre, Paul., Horowitz, Michael C. (2015). An Introduction to Autonomy in Weapon Systems.

Center for New American Security. https://s3.amazonaws.com/files.cnas.org/documents/Ethical-

Autonomy-Working-Paper_021015_v02.pdf. Accessed Nov 2016.

Shachtman, Noah. (2007). Robot Cannon Kills 9, Wounds 14. Wired Magazine.

https://www.wired.com/2007/10/robot-cannon-ki/. Accessed Nov 2016.

Sharp. (2016). Sharp INTELLOS™ A-UGV. Website. Intelligence. http://sharpintellos.com. Ac-

cessed Nov 2016.

74

Shaughnessy, Larry. (2012). One soldier, one year: $850,000 and rising. CNN.

http://security.blogs.cnn.com/2012/02/28/one-soldier-one-year-850000-and-rising/. Accessed

Dec 2016

Shobhit, Seth. (2015). Basics of Algorithmic Trading: Concepts and Examples. Investopedia.

http://www.investopedia.com/articles/active-trading/101014/basics-algorithmic-trading-

concepts-and-examples.asp. Accessed Nov 2016.

Singer, P.W., Jeffrey Lin., Hu Yu., Qian Xiaohu. (2016). China’s Army Hosts An Autonomous

Robot Contest Called “Overcoming Obstacle 2016”. Popular Science.

http://www.popsci.com/chinas-army-hosts-an-autonomous-robots-contest. Accessed Nov 2016.

Singer, P.W. Cole, August. (2015). Ghost Fleet. Houghton Mifflin Harcourt Publishing Compa-

ny. New York, New York.

Singer, P.W. (2009). In the Loop? Armed Robots and the Future of War. Brookings Institute.

https://www.brookings.edu/articles/in-the-loop-armed-robots-and-the-future-of-war/. Accessed

Nov 2016.

Singer, P.W., August Cole. (2015). The weapons the U.S. needs for a war it doesn’t want.

Reuters. http://blogs.reuters.com/great-debate/2015/07/19/the-weapons-we-need-for-a-war-we-

dont-want/. Accessed Nov 2016.

Singer, P.W. (2009). Wired for War The Robotics Revolution and Conflict in the 21st Century.

Penguin Books. New York, New York.

Singh, Abhijit. (2016). Is China Really Building Missiles With Artificial Intelligence? There are

real limits on the amount of AI acceptable to navy commanders. The Diplomat.

http://thediplomat.com/2016/09/is-china-really-building-missiles-with-artificial-intelligence/.

Accessed Nov 2016.


lease. Information & Computing Sciences. SRI International.

https://www.sri.com/newsroom/press-releases/siri-launches-virtual-personal-assistant-iphone-

3gs. Accessed Nov 2016.

Stanford Program in AI-assisted Care (PAC). (2016). Stanford Program in AI-assisted Care

(PAC) Website. Stanford University. http://vision.stanford.edu/pac/. Accessed Nov 2016.

Strain, Steve., Sean Kugele., Stan Franklin. (2014). The Learning Intelligent Distribution Agent

(LIDA) and Medical Agent X (MAX): Computational Intelligence for Medical Diagnosis Cogni-

tive Computing Research Group (CCRG). University of Memphis. Memphis, Tennessee USA.

http://ccrg.cs.memphis.edu/assets/papers/2014/PID3418857.pdf. Accessed Nov 2016.

Stout, Kristie Lu. (2016).Cyber warfare: Who is China hacking now?. CNN.

http://www.cnn.com/2016/09/29/asia/china-cyber-spies-hacking/. Accessed Nov 2016.

http://security.blogs.cnn.com/2012/02/28/one-soldier-one-year-850000-and-rising/

75

Turing, Alan. (1950). Computing Machinery and Intelligence. Mind.

http://loebner.net/Prizef/TuringArticle.html. Accessed Nov 2016.

The United States Marine Corps. (2016). The Marine Corps Operating Concept How an Expedi-

tionary Force Operates in the 21st Century. Marine Corps Combat Development Command.

http://www.mccdc.marines.mil/Portals/172/Docs/MCCDC/MOC/Marine%20Corps%20Operatin

g%20Concept%20Sept%202016.pdf?ver=2016-09-28-084156-190. accessed October 2016.

Quantico Virginia.

United Nations Office at Geneva. (2016). 2016 Meeting of Experts on LAWS. United Nations.

http://www.unog.ch/80256EE600585943/(httpPages)/37D51189AC4FB6E1C1257F4D004CAF

B2?OpenDocument. Accessed Nov 2016.

United Nations. (2008). Responsibility of States for International Wrongful Acts 2001. Interna-

tional Law Commission. United Nations.

http://legal.un.org/ilc/texts/instruments/english/draft_articles/9_6_2001.pdf. Accessed Nov 2016.

United Nations. (2002). A/RES/56/83 Responsibility of States for internationally wrongful acts

12 Dec. 2001. https://www.un.org/en/ga/search/view_doc.asp?symbol=A/RES/56/83. Accessed

Nov 2016.

United Nations. (2004). UN Report of the High-Level Panel on Threats, Challenges and Change.

"A More Secure World: Our Shared Responsibility.” UN General Assembly. A/59/565.

The University of Memphis. Projects: IDA and LIDA. Institute for Intelligent Systems. The Uni-

versity of Memphis. http://www.memphis.edu/iis/projects/ida_and_lida.php. Accessed Nov

2016.

The Vision Lab. (2016). The Vision Lab Home Page. The Vision Lab. Computer Science De-

partment. Stanford University. http://vision.stanford.edu/index.html. Accessed Nov 2017.

Wada, Kazuyoshi., Shibata, Takanori., Saito, Tomoko., Tanie, Kazuo. (2004). Effects of Robot-

Assisted Activity for Elderly People and Nurses at a Day Service Center. Invited Paper. Proceed-

ing of the IEEE. Vol. 92. No 11. Page 1780-1788.

Wolens, Doug. (2009). Singularity 101 with Vernor Vinge. Humanity Plus Magazine.

http://hplusmagazine.com/2009/04/22/singularity-101-vernor-vinge/. Accessed Nov 2016.

Work, Robert. (2015). The Third U.S. Offset Strategy and its Implications for Partners and Al-

lies. Deputy Secretary of Defense Speech. As Delivered by Deputy Secretary of Defense Bob

Work, Willard Hotel, Washington, D.C.. http://www.defense.gov/News/Speeches/Speech-

View/Article/606641/the-third-us-offset-strategy-and-its-implications-for-partners-and-allies.

Accessed Nov 2016.

76

Work, Robert. & Brimley, Shawn. (2014). 20YY Preparing for War in the Robotic Age. Center

for a New American Security.

https://s3.amazonaws.com/files.cnas.org/documents/CNAS_20YY_WorkBrimley.pdf. Accessed

October 2016.

Yan, Jie. (2016). China’s Courts Look to AI for Smarter Judgments Big data and machine learn-

ing to make China’s judicial system more intelligent and efficient. Sixth Tone.

http://www.sixthtone.com/news/china’s-courts-look-ai-smarter-judgments. Accessed Nov 2016.

Yuanchao, Li., Liu Yandong., Han Qide. (2015). Robot Products from Our School were

Unveiled at the World Robot Conference. Harbin Institute of Technology.

http://en.hit.edu.cn/snews.asp?id=1001. Accessed Nov 2016.

Yudkowsky, Eliezer. (2008). Artificial Intelligence as a Positive and Negative Factor in Global

Risk. Machine Intelligence Research Institute. http://intelligence.org/files/AIPosNegFactor.pdf.

Accessed Nov 2016.

Yudkowsky, Eliezer. (2016). Box Experiment Webpage. Yudkowski Website.

http://www.yudkowsky.net/singularity/aibox/. Accessed Nov 2016.

Yudkowsky, Eliezer. (2004). Coherent Extrapolated Volition. Machine Intelligence Research

Institute. https://intelligence.org/files/CEV.pdf. Accessed Nov 2016.

Zipline. (2016). The Future of Healthcare is Out for Delivery. Zipline Homepage.

http://flyzipline.com/product/. Accessed Nov 2016.

Major Ted W. Schroeder is an infantry officer in the United States Marine Corps currently sta-

tioned in Madison Wisconsin. He recently completed is master’s degree in International Service

at American University.