Algorithmic moral control of war robots : philosophical questions

Research output: Journal PublicationsJournal Article (refereed)Researchpeer-review

6 Citations (Scopus)

Abstract

In a series of publications, Ronald Arkin and his team have proposed the concept of a deontologically programmed 'ethical governor,' which is supposed to effectively control and enforce the ethical use of lethal force by robots on the battlefield. This paper attempts to analyse the concept of an ethical governor in the context of a more general criticism of algorithmic robot morality implementations. It is argued that the metaphor of the ethical governor is dangerously misleading in multiple respects: the governor, as proposed by Arkin, overlooks a fundamental clash of interests of the robot designer/operator, that is not present in the original governor, and that can be shown to make effective robot control in the proposed implementation impossible. The concept also suggests that ethics control is a matter of correcting behavioural deviations from a 'reference ethical action' by a negative feedback loop, although it can be shown that this does not lead to an appropriate description of moral behaviour, and that in particular it overlooks the central role of conscience and dissent in morality. Finally, the concept as proposed is based on a fundamental confusion of the properties of laws, rules of just war, terms of engagement, and moral rules. At the same time, experimental implementations of 'moral' robot controllers threaten to produce an ad-hoc regulation of ethical issues on the battlefield, which is removed from public scrutiny and democratic control. Considering these issues, the concept of an ethical governor, and, more generally, the existing attempts to handle robot morality by algorithmic means can be shown to be both misleading and dangerous, and to not appropriately address the moral problems they are supposed to solve. Consequently, such attempts in their present form must be questioned and re-examined, and a more critical approach to artefact morality should be adopted.
Original languageEnglish
Pages (from-to)279-301
Number of pages23
JournalLaw, Innovation and Technology
Volume3
Issue number2
DOIs
Publication statusPublished - 1 Dec 2011

Fingerprint

Governors
robot
Ethics
Robots
morality
Ethical Analysis
constitutional state
Confusion
Metaphor
Dissent and Disputes
Artifacts
Publications
present
conscience
Warfare
metaphor
artifact
criticism
moral philosophy
Feedback

Keywords

  • Arkin
  • Ethical governor
  • Moral control
  • Robot ethics
  • War robots

Cite this

@article{d8ffab44842c4d46b1a6e78d7956db2d,
title = "Algorithmic moral control of war robots : philosophical questions",
abstract = "In a series of publications, Ronald Arkin and his team have proposed the concept of a deontologically programmed 'ethical governor,' which is supposed to effectively control and enforce the ethical use of lethal force by robots on the battlefield. This paper attempts to analyse the concept of an ethical governor in the context of a more general criticism of algorithmic robot morality implementations. It is argued that the metaphor of the ethical governor is dangerously misleading in multiple respects: the governor, as proposed by Arkin, overlooks a fundamental clash of interests of the robot designer/operator, that is not present in the original governor, and that can be shown to make effective robot control in the proposed implementation impossible. The concept also suggests that ethics control is a matter of correcting behavioural deviations from a 'reference ethical action' by a negative feedback loop, although it can be shown that this does not lead to an appropriate description of moral behaviour, and that in particular it overlooks the central role of conscience and dissent in morality. Finally, the concept as proposed is based on a fundamental confusion of the properties of laws, rules of just war, terms of engagement, and moral rules. At the same time, experimental implementations of 'moral' robot controllers threaten to produce an ad-hoc regulation of ethical issues on the battlefield, which is removed from public scrutiny and democratic control. Considering these issues, the concept of an ethical governor, and, more generally, the existing attempts to handle robot morality by algorithmic means can be shown to be both misleading and dangerous, and to not appropriately address the moral problems they are supposed to solve. Consequently, such attempts in their present form must be questioned and re-examined, and a more critical approach to artefact morality should be adopted.",
keywords = "Arkin, Ethical governor, Moral control, Robot ethics, War robots",
author = "Andreas MATTHIAS",
year = "2011",
month = "12",
day = "1",
doi = "10.5235/175799611798204923",
language = "English",
volume = "3",
pages = "279--301",
journal = "Law, Innovation and Technology",
issn = "1757-9961",
publisher = "Taylor and Francis Ltd.",
number = "2",

}

Algorithmic moral control of war robots : philosophical questions. / MATTHIAS, Andreas.

In: Law, Innovation and Technology, Vol. 3, No. 2, 01.12.2011, p. 279-301.

Research output: Journal PublicationsJournal Article (refereed)Researchpeer-review

TY - JOUR

T1 - Algorithmic moral control of war robots : philosophical questions

AU - MATTHIAS, Andreas

PY - 2011/12/1

Y1 - 2011/12/1

N2 - In a series of publications, Ronald Arkin and his team have proposed the concept of a deontologically programmed 'ethical governor,' which is supposed to effectively control and enforce the ethical use of lethal force by robots on the battlefield. This paper attempts to analyse the concept of an ethical governor in the context of a more general criticism of algorithmic robot morality implementations. It is argued that the metaphor of the ethical governor is dangerously misleading in multiple respects: the governor, as proposed by Arkin, overlooks a fundamental clash of interests of the robot designer/operator, that is not present in the original governor, and that can be shown to make effective robot control in the proposed implementation impossible. The concept also suggests that ethics control is a matter of correcting behavioural deviations from a 'reference ethical action' by a negative feedback loop, although it can be shown that this does not lead to an appropriate description of moral behaviour, and that in particular it overlooks the central role of conscience and dissent in morality. Finally, the concept as proposed is based on a fundamental confusion of the properties of laws, rules of just war, terms of engagement, and moral rules. At the same time, experimental implementations of 'moral' robot controllers threaten to produce an ad-hoc regulation of ethical issues on the battlefield, which is removed from public scrutiny and democratic control. Considering these issues, the concept of an ethical governor, and, more generally, the existing attempts to handle robot morality by algorithmic means can be shown to be both misleading and dangerous, and to not appropriately address the moral problems they are supposed to solve. Consequently, such attempts in their present form must be questioned and re-examined, and a more critical approach to artefact morality should be adopted.

AB - In a series of publications, Ronald Arkin and his team have proposed the concept of a deontologically programmed 'ethical governor,' which is supposed to effectively control and enforce the ethical use of lethal force by robots on the battlefield. This paper attempts to analyse the concept of an ethical governor in the context of a more general criticism of algorithmic robot morality implementations. It is argued that the metaphor of the ethical governor is dangerously misleading in multiple respects: the governor, as proposed by Arkin, overlooks a fundamental clash of interests of the robot designer/operator, that is not present in the original governor, and that can be shown to make effective robot control in the proposed implementation impossible. The concept also suggests that ethics control is a matter of correcting behavioural deviations from a 'reference ethical action' by a negative feedback loop, although it can be shown that this does not lead to an appropriate description of moral behaviour, and that in particular it overlooks the central role of conscience and dissent in morality. Finally, the concept as proposed is based on a fundamental confusion of the properties of laws, rules of just war, terms of engagement, and moral rules. At the same time, experimental implementations of 'moral' robot controllers threaten to produce an ad-hoc regulation of ethical issues on the battlefield, which is removed from public scrutiny and democratic control. Considering these issues, the concept of an ethical governor, and, more generally, the existing attempts to handle robot morality by algorithmic means can be shown to be both misleading and dangerous, and to not appropriately address the moral problems they are supposed to solve. Consequently, such attempts in their present form must be questioned and re-examined, and a more critical approach to artefact morality should be adopted.

KW - Arkin

KW - Ethical governor

KW - Moral control

KW - Robot ethics

KW - War robots

UR - http://commons.ln.edu.hk/sw_master/757

U2 - 10.5235/175799611798204923

DO - 10.5235/175799611798204923

M3 - Journal Article (refereed)

VL - 3

SP - 279

EP - 301

JO - Law, Innovation and Technology

JF - Law, Innovation and Technology

SN - 1757-9961

IS - 2

ER -