Circuit Breaking: Removing Model Behaviors with Targeted Ablation

2023-09-12 · Maximilian Li, Xander Davies, Max Nadeau

Language models often exhibit behaviors that improve performance on a pre-training objective but harm performance on downstream tasks. We propose a novel approach to removing undesirable behaviors by ablating a small number of causal pathways between model components, with the intention of disabling the computational circuit responsible for the bad behavior. Given a small dataset of inputs where the model behaves poorly, we learn to ablate a small number of important causal pathways. In the setting of reducing GPT-2 toxic language generation, we find ablating just 12 of the 11.6K causal edges mitigates toxic generation with minimal degradation of performance on other inputs.

📄 PDF Abstract BibTeX arXiv:2309.05973

Code (1)

xanderdavies/circuit-breaking 공식 구현 pytorch

Tasks

modelText Generation

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Discriminative Fine-Tuning Discriminative Fine-Tuning is a fine-tuning strategy that is used for ULMFiT type models. Instead of using the same learning rate…

Residual Connection 설명 없음

Adam 설명 없음

Weight Decay 설명 없음

ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attack

2025-09-30 · Yein Park, Jungwoo Park, Jaewoo Kang arxiv

Large language models (LLMs), despite being safety-aligned, exhibit brittle refusal behaviors that can be circumvented by simple linguistic changes. As tense jailbreaking demonstrates that models refusing harmful request…

Attribution-guided Pruning for Compression, Circuit Discovery, and Targeted Correction in LLMs

2025-06-16 · Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Reduan Achtibat, Patrick Kahardipraja 외

Large Language Models (LLMs) are central to many contemporary AI applications, yet their extensive parameter counts pose significant challenges for deployment in memory- and compute-constrained environments. Recent works…

Model Compression

Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints

2025-02-25 · Junxiao Yang, Zhexin Zhang, Shiyao Cui, Hongning Wang 외

Jailbreaking attacks can effectively induce unsafe behaviors in Large Language Models (LLMs); however, the transferability of these attacks across different models remains limited. This study aims to understand and enhan…

Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection

2026-05-28 · Syafiq Al Atiiq, Chun Zhou, Christian Gehrmann arxiv

Large language models (LLMs) can detect software vulnerabilities, but how do they actually identify vulnerable code? We address this question using mechanistic interpretability; analyzing the internal computations of a n…

Vulnerability Detection

Unveiling Language Skills via Path-Level Circuit Discovery

2024-10-02 · Hang Chen, Jiaying Zhu, Xinyu Yang, Wenya Wang

Circuit discovery with edge-level ablation has become a foundational framework for mechanism interpretability of language models. However, its focus on individual edges often overlooks the sequential, path-level causal r…

Disentanglement In-Context Learning Language Modelling