REBAR: Reference Ethical Benchmark for Autonomy Readiness
As autonomous systems grow more advanced, objective metrics to evaluate their ethical and legal compliance are critical for informing end users of their limitations and ensuring accountability of those who misuse them. Current ethical embodied AI frameworks remain mostly qualitative, focusing on system design (through safety guardrails or targeted red teaming), and the realized guardrails often directly disallow unsafe behavior without providing the user with an override or interpretable reason. Instead, there is a need for computable metrics through rigorous testing that allow a user to determine the applicability of the system to the task. To address this gap, we introduce the Reference Ethical Benchmark for Autonomy Readiness (REBAR), a quantitative test and evaluation framework for autonomous systems. REBAR maps operating metrics into a computable Autonomy Readiness Level (ARL) rubric that can quantify ethical performance. Key innovations of the framework include a neuro-symbolic Large Language Model (LLM) approach to calculate and explain the ethical difficulty of scenarios, LLM-driven at-scale generation of test instances, and a versatile, photorealistic simulation environment. By evaluating white-box autonomy solutions through this rigorous testing pipeline, REBAR delivers an objective and repeatable benchmark score, bridging the gap between abstract principles and verifiable, accountable autonomy.
Code (0)
등록된 구현이 없습니다.
Tasks
Red TeamingSimilar Papers 제목 키워드 기반
Autonomy for Older Adult-Agent Interaction
As the global population ages, artificial intelligence (AI)-powered agents have emerged as potential tools to support older adults' caregiving. Prior research has explored agent autonomy by identifying key interaction st…
What Does the AI Doctor Value? Auditing Pluralism in the Clinical Ethics of Language Models
Medicine is inherently pluralistic. Principles such as autonomy, beneficence, nonmaleficence, and justice routinely conflict, and such ethical dilemmas often sharply divide reasonable physicians. Good clinical practice n…
An Improved Convolutional Neural Network System for Automatically Detecting Rebar in GPR Data
As a mature technology, Ground Penetration Radar (GPR) is now widely employed in detecting rebar and other embedded elements in concrete structures. Manually recognizing rebar from GPR data is a time-consuming and error-…
GPRTruly Autonomous Machines Are Ethical
While many see the prospect of autonomous machines as threatening, autonomy may be exactly what we want in a superintelligent machine. There is a sense of autonomy, deeply rooted in the ethical literature, in which an au…
EthicsPredicting Rebar Endpoints using Sin Exponential Regression Model
Currently, unmanned automation studies are underway to minimize the loss rate of rebar production and the time and accuracy of calibration when producing defective products in the cutting process of processing rebar fact…
modelPositionPredictionregression