paper-with-me

Papers

Agentic Moderation: Multi-Agent Design for Safer Vision-Language Models

2025-10-29 · Juan Ren, Mark Dras, Usman Naseem arxiv

Agentic methods have emerged as a powerful and autonomous paradigm that enhances reasoning, collaboration, and adaptive control, enabling systems to coordinate and independently solve complex tasks. We extend this paradigm to safety alignment by introducing Agentic Moderation, a model-agnostic framework that leverages specialised agents to defend multimodal systems against jailbreak attacks. Unlike prior approaches that apply as a static layer over inputs or outputs and provide only binary classifications (safe or unsafe), our method integrates dynamic, cooperative agents, including Shield, Responder, Evaluator, and Reflector, to achieve context-aware and interpretable moderation. Extensive experiments across five datasets and four representative Large Vision-Language Models (LVLMs) demonstrate that our approach reduces the Attack Success Rate (ASR) by 7-19%, maintains a stable Non-Following Rate (NF), and improves the Refusal Rate (RR) by 4-20%, achieving robust, interpretable, and well-balanced safety performance. By harnessing the flexibility and reasoning capacity of agentic architectures, Agentic Moderation provides modular, scalable, and fine-grained safety enforcement, highlighting the broader potential of agentic systems as a foundation for automated safety governance.

📄 PDF Abstract BibTeX arXiv:2510.25179

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Value-Aligned Prompt Moderation via Zero-Shot Agentic Rewriting for Safe Image Generation

2025-11-12 · Xin Zhao, Xiaojun Chen, Bingshan Liu, Zeyao Liu 외 arxiv

Generative vision-language models like Stable Diffusion demonstrate remarkable capabilities in creative media synthesis, but they also pose substantial risks of producing unsafe, offensive, or culturally inappropriate co…

Text-to-Image Generation

Safer Conversational AI as a Source of User Delight

2023-04-18 · Xiaoding Lu, Aleksey Korshuk, Zongyi Liu, William Beauchamp 외

This work explores the impact of moderation on users' enjoyment of conversational AI systems. While recent advancements in Large Language Models (LLMs) have led to highly capable conversational AIs that are increasingly …

Safe Guard: an LLM-agent for Real-time Voice-based Hate Speech Detection in Social Virtual Reality

2024-09-23 · Yiwen Xu, Qinyang Hou, Hongyu Wan, Mirjana Prpa

In this paper, we present Safe Guard, an LLM-agent for the detection of hate speech in voice-based interactions in social VR (VRChat). Our system leverages Open AI GPT and audio feature extraction for real-time voice int…

Hate Speech Detection

AI-Native Network Controller: A Modular Framework for Safe Agentic Control of Multi-Domain Network Infrastructure

2026-04-20 · Merim Dzaferagic arxiv

The convergence of multiple network domains, including radio access, optical transport, and core networks, under unified intelligent control is a fundamental requirement for future 6G systems. This is important because e…

The Cold-Start Safety Gap in LLM Agents

2026-06-05 · Chung-En Sun, Linbo Liu, Tsui-Wei Weng arxiv

Are tool-calling LLM agents equally safe throughout a conversation? We discover they are not: agents are most vulnerable at the very start of a session and become substantially safer after a few regular agentic tasks -- …