Adapting Probabilistic Risk Assessment for AI
Modern general-purpose artificial intelligence (AI) systems present an urgent risk management challenge, as their rapidly evolving capabilities and potential for catastrophic harm outpace our ability to reliably assess their risks. Current methods often rely on selective testing and undocumented assumptions about risk priorities, frequently failing to make a serious attempt at assessing the set of pathways through which Al systems pose direct or indirect risks to society and the biosphere. This paper introduces the probabilistic risk assessment (PRA) for AI framework, adapting established PRA techniques from high-reliability industries (e.g., nuclear power, aerospace) for the new challenges of advanced AI. The framework guides assessors in identifying potential risks, estimating likelihood and severity, and explicitly documenting evidence, underlying assumptions, and analyses at appropriate granularities. The framework's implementation tool synthesizes the results into a risk report card with aggregated risk estimates from all assessed risks. This systematic approach integrates three advances: (1) Aspect-oriented hazard analysis provides systematic hazard coverage guided by a first-principles taxonomy of AI system aspects (e.g. capabilities, domain knowledge, affordances); (2) Risk pathway modeling analyzes causal chains from system aspects to societal impacts using bidirectional analysis and incorporating prospective techniques; and (3) Uncertainty management employs scenario decomposition, reference scales, and explicit tracing protocols to structure credible projections with novelty or limited data. Additionally, the framework harmonizes diverse assessment methods by integrating evidence into comparable, quantified absolute risk estimates for critical decisions. We have implemented this as a workbook tool for AI developers, evaluators, and regulators, available on the project website.
Code (0)
등록된 구현이 없습니다.
Tasks
ManagementMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Simulation-based Probabilistic Risk Assessment
Simulation-based probabilistic risk assessment (SPRA) is a systematic and comprehensive methodology that has been used and refined over the past few decades to evaluate the risks associated with complex systems. SPRA mod…
PRREACH: Probabilistic Risk Assessment Using Reachability for UAV Control
We present a new approach for designing risk-bounded controllers for Uncrewed Aerial Vehicles (UAVs). Existing frameworks for assessing risk of UAV operations rely on knowing the conditional probability of an incident oc…
Spectral risk measures and uncertainty
Risk assessment under different possible scenarios is a source of uncertainty that may lead to concerning financial losses. We address this issue, first, by adapting a robust framework to the class of spectral risk measu…
A Machine-learning based Probabilistic Perspective on Dynamic Security Assessment
Probabilistic security assessment and real-time dynamic security assessments (DSA) are promising to better handle the risks of system operations. The current methodologies of security assessments may require many time-do…
BIG-bench Machine LearningProbabilistic Risk Assessment of an Obstacle Detection System for GoA 4 Freight Trains
In this paper, a quantitative risk assessment approach is discussed for the design of an obstacle detection function for low-speed freight trains with grade of automation (GoA)~4. In this 5-step approach, starting with s…