Detecting Safety Violations Across Many Agent Traces

2026-04-13 · Adam Stein, Davis Brown, Hamed Hassani, Mayur Naik, Eric Wong arxiv

To identify safety violations, auditors often search over large sets of agent traces. This search is difficult because failures are often rare, complex, and sometimes even adversarially hidden and only detectable when multiple traces are analyzed together. These challenges arise in diverse settings such as misuse campaigns, covert sabotage, reward hacking, and prompt injection. Existing approaches struggle here for several reasons. Per-trace judges miss failures that only become visible across traces, naive agentic auditing does not scale to large trace collections, and fixed monitors are brittle to unanticipated behaviors. We introduce Meerkat, which combines clustering with agentic search to uncover violations specified in natural language. Through structured search and adaptive investigation of promising regions, Meerkat finds sparse failures without relying on seed scenarios, fixed workflows, or exhaustive enumeration. Across misuse, misalignment, and task gaming settings, Meerkat significantly improves detection of safety violations over baseline monitors, discovers widespread developer cheating on a top agent benchmark, and finds nearly 4x more examples of reward hacking on CyBench than previous audits.

📄 PDF Abstract BibTeX arXiv:2604.11806

Code (0)

등록된 구현이 없습니다.

Auditing Agent Harness Safety

2026-05-14 · Chengzhi Liu, Yichen Guo, Yepeng Liu, Yuzhe Yang 외 arxiv

LLM agents increasingly run inside execution harnesses that dispatch tools, allocate resources, and route messages between specialized components. However, a harness can return a correct, benign answer over a trajectory …

OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows

2025-10-28 · Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie 외 arxiv

Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms. While these agents hold great promise for advancing digita…

Automated detection of atomicity violations in large-scale systems

2025-04-01 · Hang He, Yixing Luo, Chengcheng Wan, Ting Su 외

Atomicity violations in interrupt-driven programs pose a significant threat to software safety in critical systems. These violations occur when the execution sequence of operations on shared resources is disrupted by asy…

Language Modeling Language Modelling Large Language Model

DESTA: A Framework for Safe Reinforcement Learning with Markov Games of Intervention

2021-10-27 · David Mguni, Usman Islam, Yaqi Sun, Xiuling Zhang 외

Reinforcement learning (RL) involves performing exploratory actions in an unknown system. This can place a learning agent in dangerous and potentially catastrophic system states. Current approaches for tackling safe lear…

OpenAI Gym reinforcement-learning Reinforcement Learning (RL)Safe Exploration+1

Forecasting Trajectory-Level Safety Risks in Black-Box Multi-Turn Interactions

2026-07-29 · Shi Lin, Peng Qian, Dinghao Liu, Renjie Sun 외 arxiv

As large language models (LLMs) evolve from standalone assistants into autonomous agents, ensuring their safety requires shifting beyond pointwise risk assessment to understand how risks emerge and unfold over long-horiz…