AI Alignment Versus AI Ethical Treatment : 10 Challenges

Adam BRADLEY; Bradford SAAD

doi:10.1111/phib.12380

AI Alignment Versus AI Ethical Treatment : 10 Challenges

Adam BRADLEY, Bradford SAAD^*

^*Corresponding author for this work

Lingnan University

Research output: Journal Publications › Journal Article (refereed) › peer-review

Abstract

A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment.

Original language	English
Journal	Analytic Philosophy
Early online date	11 Aug 2025
DOIs	https://doi.org/10.1111/phib.12380
Publication status	E-pub ahead of print - 11 Aug 2025

Bibliographical note

For helpful feedback, thanks to Adam Bales, Nick Bostrom, Patrick Butlin, Tushita Jha, Kyle Fish, Robert Long, Kritika Maheshwari, Andreas Mogensen, Steve Petersen, Eric Schwitzgebel, Derek Shiller, Carl Shulman, Jonathan Simon, Rhys Southan, Elliott Thornley, and participants at the The Mimir Center's 2025 Workshop on the Ethics and Philosophy of Brain Emulation, and the 2025 Chalmers University of Technology AI Ethics Seminar, as well as an audience at Lingnan University.

Publisher Copyright: © 2025 The Author(s). Analytic Philosophy published by John Wiley & Sons Ltd.

Publisher Copyright:
© 2025 The Author(s). Analytic Philosophy published by John Wiley & Sons Ltd.

Funding

Such projects have been supported by Australia, China, the European Union, Japan, South Korea, and the United States (Naddaf 2023 ).

Keywords

AI rights
AI safety
AI wellbeing
catastrophic risks
machine consciousness
value alignment problem
whole-brain emulation

Access to Document

10.1111/phib.12380Licence: CC BY

https://onlinelibrary.wiley.com/doi/10.1111/phib.12380

Cite this

@article{ceec412715e5437983c5829d6e8f095e,

title = "AI Alignment Versus AI Ethical Treatment : 10 Challenges",

abstract = "A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment.",

keywords = "AI rights, AI safety, AI wellbeing, catastrophic risks, machine consciousness, value alignment problem, whole-brain emulation",

author = "Adam BRADLEY and Bradford SAAD",

note = "For helpful feedback, thanks to Adam Bales, Nick Bostrom, Patrick Butlin, Tushita Jha, Kyle Fish, Robert Long, Kritika Maheshwari, Andreas Mogensen, Steve Petersen, Eric Schwitzgebel, Derek Shiller, Carl Shulman, Jonathan Simon, Rhys Southan, Elliott Thornley, and participants at the The Mimir Center's 2025 Workshop on the Ethics and Philosophy of Brain Emulation, and the 2025 Chalmers University of Technology AI Ethics Seminar, as well as an audience at Lingnan University. Publisher Copyright: {\textcopyright} 2025 The Author(s). Analytic Philosophy published by John Wiley \& Sons Ltd. Publisher Copyright: {\textcopyright} 2025 The Author(s). Analytic Philosophy published by John Wiley \& Sons Ltd.",

year = "2025",

month = aug,

day = "11",

doi = "10.1111/phib.12380",

language = "English",

journal = "Analytic Philosophy",

issn = "2153-960x",

publisher = "Wiley",

}

TY - JOUR

T1 - AI Alignment Versus AI Ethical Treatment : 10 Challenges

AU - BRADLEY, Adam

AU - SAAD, Bradford

N1 - For helpful feedback, thanks to Adam Bales, Nick Bostrom, Patrick Butlin, Tushita Jha, Kyle Fish, Robert Long, Kritika Maheshwari, Andreas Mogensen, Steve Petersen, Eric Schwitzgebel, Derek Shiller, Carl Shulman, Jonathan Simon, Rhys Southan, Elliott Thornley, and participants at the The Mimir Center's 2025 Workshop on the Ethics and Philosophy of Brain Emulation, and the 2025 Chalmers University of Technology AI Ethics Seminar, as well as an audience at Lingnan University. Publisher Copyright: © 2025 The Author(s). Analytic Philosophy published by John Wiley & Sons Ltd. Publisher Copyright: © 2025 The Author(s). Analytic Philosophy published by John Wiley & Sons Ltd.

PY - 2025/8/11

Y1 - 2025/8/11

N2 - A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment.

AB - A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment.

KW - AI rights

KW - AI safety

KW - AI wellbeing

KW - catastrophic risks

KW - machine consciousness

KW - value alignment problem

KW - whole-brain emulation

UR - https://www.scopus.com/pages/publications/105012935368

U2 - 10.1111/phib.12380

DO - 10.1111/phib.12380

M3 - Journal Article (refereed)

SN - 2153-960x

JO - Analytic Philosophy

JF - Analytic Philosophy

ER -

AI Alignment Versus AI Ethical Treatment : 10 Challenges

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this