What AI evaluations for preventing catastrophic risks can and cannot do
AI evaluations are an important component of the AI governance toolkit, underlying current approaches to safety cases for preventing catastrophic risks. Our paper examines what these evaluations can and cannot tell us. Evaluations can establish lower bounds on AI capabilities and assess certain misuse risks given sufficient effort from evaluators. Unfortunately, evaluations face fundamental limitations that cannot be overcome within the current paradigm. These include an inability to establish upper bounds on capabilities, reliably forecast future model capabilities, or robustly assess risks from autonomous AI systems. This means that while evaluations are valuable tools, we should not rely on them as our main way of ensuring AI systems are safe. We conclude with recommendations for incremental improvements to frontier AI safety, while acknowledging these fundamental limitations remain unsolved.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Muse Spark Safety & Preparedness Report
Muse Spark is the latest large language model developed by Meta. In this report, we first present evaluations for catastrophic risk domains under Meta's Advanced AI Scaling Framework, along with the evidence that informe…
Evaluating Frontier Models for Dangerous Capabilities
To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new "dangerous capability" evaluations and pilot them on Gemini 1.0 mode…
How Catastrophic is Your LLM? Certifying Risk in Conversation
Large Language Models (LLMs) can produce catastrophic responses in conversational settings that pose serious risks to public safety and security. Existing evaluations often fail to fully reveal these vulnerabilities beca…
Semantic SimilaritySTREAM (ChemBio): A Standard for Transparently Reporting Evaluations in AI Model Reports
Evaluations of dangerous AI capabilities are important for managing catastrophic risks. Public transparency into these evaluations - including what they test, how they are conducted, and how their results inform decision…
Could AI be the Great Filter? What Astrobiology can Teach the Intelligence Community about Anthropogenic Risks
Where is everybody? This phrase distills the foreboding of what has come to be known as the Fermi Paradox - the disquieting idea that, if extraterrestrial life is probable in the Universe, then why have we not encountere…