Safety is Non-Compositional: A Formal Framework for Capability-Based AI Systems
This paper contains the first formal proof that safety is non-compositional in the presence of conjunctive capability dependencies: two agents each individually inca- pable of reaching any forbidden capability can, when combined, collectively reach a forbidden goal through an emergent conjunctive dependency.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
From Data to Control: A Formal Compositional Framework for Large-Scale Interconnected Networks
We introduce a compositional data-driven methodology with noisy data for designing fully-decentralized safety controllers applicable to large-scale interconnected networks, encompassing a vast number of subsystems with u…
Falcon: Functional Assembly and Language for Compositional Reasoning in X-ray
Conventional vision-language models are largely object-centric, focusing on detecting and describing individual entities. In safety-critical X-ray baggage screening, however, threat often emerges not from a single object…
Safety by Invariance, Liveness through Refinement: Heterogeneous Contract Framework for Co-Design of Layered Control
Real-world control systems must achieve long-horizon objectives (liveness) while respecting continuous-time safety constraints, a combination that motivates hierarchical layered control architectures (LCAs). Existing LCA…
Assumption Generation for the Verification of Learning-Enabled Autonomous Systems
Providing safety guarantees for autonomous systems is difficult as these systems operate in complex environments that require the use of learning-enabled components, such as deep neural networks (DNNs) for visual percept…
Compositional Design of Safety Controllers for Large-scale Stochastic Hybrid Systems
In this work, we propose a compositional scheme based on small-gain reasoning for the safety controller synthesis of interconnected stochastic hybrid systems with both continuous evolutions and instantaneous jumps. In ou…