Large Empirical Case Study: Go-Explore adapted for AI Red Team Testing
Production LLM agents with tool-using capabilities require security testing despite their safety training. We adapt Go-Explore to evaluate GPT-4o-mini across 28 experimental runs spanning six research questions. We find that random-seed variance dominates algorithmic parameters, yielding an 8x spread in outcomes; single-seed comparisons are unreliable, while multi-seed averaging materially reduces variance in our setup. Reward shaping consistently harms performance, causing exploration collapse in 94% of runs or producing 18 false positives with zero verified attacks. In our environment, simple state signatures outperform complex ones. For comprehensive security testing, ensembles provide attack-type diversity, whereas single agents optimize coverage within a given attack type. Overall, these results suggest that seed variance and targeted domain knowledge can outweigh algorithmic sophistication when testing safety-trained models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Exploration in Model-based Reinforcement Learning with Randomized Reward
Model-based Reinforcement Learning (MBRL) has been widely adapted due to its sample efficiency. However, existing worst-case regret analysis typically requires optimistic planning, which is not realistic in general. In c…
Efficient ExplorationModel-based Reinforcement Learningreinforcement-learningReinforcement Learning+1Can Post-Training Turn LLMs into Good Medical Coders? An Empirical Study of Generative ICD Coding
Automated International Classification of Diseases (ICD) coding is a core medical-coding task for billing, epidemiology, and clinical decision support. Generative large language models (LLMs) are often reported as weak m…
Reinforcement LearningNon-Visible Light Data Synthesis and Application: A Case Study for Synthetic Aperture Radar Imagery
We explore the "hidden" ability of large-scale pre-trained image generation models, such as Stable Diffusion and Imagen, in non-visible light domains, taking Synthetic Aperture Radar (SAR) data for a case study. Due to t…
Image GenerationAdapting OCR with limited supervision
Text recognition systems of today (aka OCRs) are mostly based on supervised learning of deep neural networks. Performance of these are limited by the type of data that is used for training. In the presence of diverse st…
Optical Character Recognition (OCR)Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights
Large Language Models(LLMs) are increasingly explored for cybersecurity applications such as vulnerability detection. In the domain of threat modelling, prior work has primarily evaluated a number of general-purpose Larg…
Vulnerability DetectionDomain Adaptation