Acting for the Right Reasons: Creating Reason-Sensitive Artificial Moral Agents

2024-09-23 · Kevin Baum, Lisa Dargasz, Felix Jahn, Timo P. Gros, Verena Wolf

We propose an extension of the reinforcement learning architecture that enables moral decision-making of reinforcement learning agents based on normative reasons. Central to this approach is a reason-based shield generator yielding a moral shield that binds the agent to actions that conform with recognized normative reasons so that our overall architecture restricts the agent to actions that are (internally) morally justified. In addition, we describe an algorithm that allows to iteratively improve the reason-based shield generator through case-based feedback from a moral judge.

📄 PDF Abstract BibTeX arXiv:2409.15014

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making reinforcement-learning Reinforcement Learning

Doing the right thing for the right reason: Evaluating artificial moral cognition by probing cost insensitivity

2023-05-29 · Yiran Mao, Madeline G. Reinecke, Markus Kunesch, Edgar A. Duéñez-Guzmán 외

Is it possible to evaluate the moral cognition of complex artificial agents? In this work, we take a look at one aspect of morality: `doing the right thing for the right reasons.' We propose a behavior-based analysis of …

Deep Reinforcement Learning Meta Reinforcement Learning reinforcement-learning Reinforcement Learning

A Pragmatic View of AI Personhood

2025-10-30 · Joel Z. Leibo, Alexander Sasha Vezhnevets, William A. Cunningham, Stanley M. Bileschi arxiv

The emergence of agentic Artificial Intelligence (AI) is set to trigger a "Cambrian explosion" of new kinds of personhood. This paper proposes a pragmatic framework for navigating this diversification by treating personh…

LLMs as Assessors: Right for the Right Reason?

2026-01-13 · Sourav Saha, Mandar Mitra, Aditya Dutta arxiv

A good deal of recent research has focused on how Large Language Models (LLMs) may be used as judges in place of humans to evaluate the quality of the output produced by various text / image processing systems. Within th…

Information Retrieval

Making deep neural networks right for the right scientific reasons by interacting with their explanations

2020-01-15 · Patrick Schramowski, Wolfgang Stammer, Stefano Teso, Anna Brugger 외

Deep neural networks have shown excellent performances in many real-world applications. Unfortunately, they may show "Clever Hans"-like behavior -- making use of confounding factors within datasets -- to achieve high per…

BIG-bench Machine Learning Plant Phenotyping

Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting

2020-10-04 · ICLR 2021 1 · Sayna Ebrahimi, Suzanne Petryk, Akash Gokul, William Gan 외

The goal of continual learning (CL) is to learn a sequence of tasks without suffering from the phenomenon of catastrophic forgetting. Previous work has shown that leveraging memory in the form of a replay buffer can redu…

Continual Learning