Lifelong Safety Alignment for Language Models

2025-05-26 · Haoyu Wang, Zeyu Qin, Yifei Zhao, Chao Du, Min Lin, Xueqian Wang, Tianyu Pang

LLMs have made impressive progress, but their growing capabilities also expose them to highly flexible jailbreaking attacks designed to bypass safety alignment. While many existing defenses focus on known types of attacks, it is more critical to prepare LLMs for unseen attacks that may arise during deployment. To address this, we propose a lifelong safety alignment framework that enables LLMs to continuously adapt to new and evolving jailbreaking strategies. Our framework introduces a competitive setup between two components: a Meta-Attacker, trained to actively discover novel jailbreaking strategies, and a Defender, trained to resist them. To effectively warm up the Meta-Attacker, we first leverage the GPT-4o API to extract key insights from a large collection of jailbreak-related research papers. Through iterative training, the first iteration Meta-Attacker achieves a 73% attack success rate (ASR) on RR and a 57% transfer ASR on LAT using only single-turn attacks. Meanwhile, the Defender progressively improves its robustness and ultimately reduces the Meta-Attacker's success rate to just 7%, enabling safer and more reliable deployment of LLMs in open-ended environments. The code is available at https://github.com/sail-sg/LifelongSafetyAlignment.

📄 PDF Abstract BibTeX arXiv:2505.20259

Code (1)

sail-sg/lifelongsafetyalignment 공식 구현 pytorch

Tasks

Safety Alignment

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Narrow Fine-Tuning Erodes Safety Alignment in Vision-Language Agents

2026-02-18 · Idhant Gulati, Shivam Raval arxiv

Lifelong multimodal agents must continuously adapt to new tasks through post-training, but this creates a fundamental tension between acquiring capabilities and preserving safety alignment. We demonstrate that fine-tunin…

Continual Learning

LifeAlign: Lifelong Alignment for Large Language Models with Memory-Augmented Focalized Preference Optimization

2025-09-21 · Junsong Li, Jie Zhou, Bihao Zhan, Yutao Yang 외 arxiv

Alignment plays a crucial role in Large Language Models (LLMs) in aligning with human preferences on a specific task/domain. Traditional alignment methods suffer from catastrophic forgetting, where models lose previously…

Dimensionality Reduction

AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection

2025-02-17 · Weidi Luo, Shenghong Dai, Xiaogeng Liu, Suman Banerjee 외

The rapid advancements in Large Language Models (LLMs) have enabled their deployment as autonomous agents for handling complex tasks in dynamic environments. These LLMs demonstrate strong problem-solving capabilities and…

Safe Policy Search for Lifelong Reinforcement Learning with Sublinear Regret

2015-05-21 · Haitham Bou Ammar, Rasul Tutunov, Eric Eaton

Lifelong reinforcement learning provides a promising framework for developing versatile agents that can accumulate knowledge over a lifetime of experience and rapidly learn new tasks by building upon prior knowledge. How…

Lifelong learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs

2026-06-11 · He Li, Haoang Chi, Qizhou Wang, Yunxin Mao 외 arxiv

Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often…