paper-with-me

홈 › Papers

Moral Uncertainty and the Problem of Fanaticism

2023-12-18 · Jazon Szabo, Jose Such, Natalia Criado, Sanjay Modgil

While there is universal agreement that agents ought to act ethically, there is no agreement as to what constitutes ethical behaviour. To address this problem, recent philosophical approaches to moral uncertainty' propose aggregation of multiple ethical theories to guide agent behaviour. However, one of the foundational proposals for aggregation - Maximising Expected Choiceworthiness (MEC) - has been criticised as being vulnerable to fanaticism; the problem of an ethical theory dominating agent behaviour despite low credence (confidence) in said theory. Fanaticism thus undermines the democratic' motivation for accommodating multiple ethical perspectives. The problem of fanaticism has not yet been mathematically defined. Representing moral uncertainty as an instance of social welfare aggregation, this paper contributes to the field of moral uncertainty by 1) formalising the problem of fanaticism as a property of social welfare functionals and 2) providing non-fanatical alternatives to MEC, i.e. Highest k-trimmed Mean and Highest Median.

📄 PDF Abstract BibTeX arXiv:2312.11589

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Dropouts in Confidence: Moral Uncertainty in Human-LLM Alignment

2025-11-17 · Jea Kwon, Luiz Felipe Vecchietti, Sungwon Park, Meeyoung Cha arxiv

Humans display significant uncertainty when confronted with moral dilemmas, yet the extent of such uncertainty in machines and AI agents remains underexplored. Recent studies have confirmed the overly confident tendencie…

Accounting for Context: Shaping Moral Credences for Value Alignment

2026-06-05 · Jazon Szabo, Sanjay Modgil arxiv

Ensuring that agent behaviours are aligned with human moral values inevitably raises the problem of how to account for the plurality of moral perspectives that societies -- and even individuals -- typically adopt. Work o…

Decision Making

Uncertain Machine Ethics Planning

2025-05-07 · Simon Kolker, Louise A. Dennis, Ramon Fraga Pereira, Mengwei Xu

Machine Ethics decisions should consider the implications of uncertainty over decisions. Decisions should be made over sequences of actions to reach preferable outcomes long term. The evaluation of outcomes, however, may…

Ethics

Reinforcement Learning Under Moral Uncertainty

2020-06-08 · Adrien Ecoffet, Joel Lehman

An ambitious goal for machine learning is to create agents that behave ethically: The capacity to abide by human moral norms would greatly expand the context in which autonomous agents could be practically and safely dep…

Autonomous VehiclesBIG-bench Machine LearningPhilosophyreinforcement-learning+2

Towards Theory-based Moral AI: Moral AI with Aggregating Models Based on Normative Ethical Theory

2023-06-20 · Masashi Takeshita, Rzepka Rafal, Kenji Araki

Moral AI has been studied in the fields of philosophy and artificial intelligence. Although most existing studies are only theoretical, recent developments in AI have made it increasingly necessary to implement AI with m…

EthicsPhilosophy