MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate

2026-09-04 · Garvit Joshi, Stavya Dhyani, Jasmine, Arun Chauhan arxiv

Media bias in news articles operates through subtle linguistic cues---loaded language, selective framing, and strategic omission---that resist single-model detection and have traditionally required large annotated corpora for supervised training. We ask whether structured multi-agent deliberation can serve as a principled, training-free alternative to supervised classification for this task. We introduce MABPD (Multi-Agent Bias Probing & Detection), a pipeline in which three specialized LLM agents analyze an article from complementary perspectives and resolve disagreements through a Structured Argument Debate (SAD) protocol. SAD implements a domain-motivated asymmetric burden of proof---biased claims without grounded textual evidence carry zero weight---combined with role-weighted voting and post-consensus verification, replacing task-specific supervised decision boundaries with explicit deliberative structure. Ablation confirms that this structured deliberation, not mere agent parallelism, drives performance: removing the debate module reduces F1 by up to 10.6 points. On the BABE benchmark (4,121 expert-annotated sentences), MABPD achieves 83.4% macro F1 on the held-out test split---within 0.7 percentage points (pp) of the supervised SOTA (MAGPIE, 84.1% macro F1; Horych et al., 2024)---without any task-specific training or threshold tuning on annotated data. Cross-dataset evaluation on the SemEval 2019 HyperPartisan corpus (644 articles) yields 75.0% zero-shot accuracy, within 7.2 pp of the supervised SOTA accuracy (82.2%; Kiesel et al. 2019), confirming transfer across annotation regimes. We release the full pipeline and evaluation code.

📄 PDF Abstract BibTeX arXiv:2609.04841

Code (0)

등록된 구현이 없습니다.

Detecting Multi-Agent Collusion Through Multi-Agent Interpretability

2026-04-01 · Aaron Rose, Carissa Cullen, Sahar Abdelnabi, Philip Torr 외 arxiv

As LLM agents are increasingly deployed in multi-agent systems, they introduce risks of covert coordination that may evade standard forms of human oversight. While linear probes on model activations have shown promise fo…

Anomaly Detection

Language-Agnostic Bias Detection in Language Models with Bias Probing

2023-05-22 · Abdullatif Köksal, Omer Faruk Yalcin, Ahmet Akbiyik, M. Tahir Kilavuz 외

Pretrained language models (PLMs) are key components in NLP, but they contain strong social biases. Quantifying these biases is challenging because current methods focusing on fill-the-mask objectives are sensitive to sl…

Bias Detection

Unbiased Evaluation of Large Language Models from a Causal Perspective

2025-02-10 · Meilin Chen, Jian Tian, Liang Ma, Di Xie 외

Benchmark contamination has become a significant concern in the LLM evaluation community. Previous Agents-as-an-Evaluator address this issue by involving agents in the generation of questions. Despite their success, the …

Agent Probing Interaction Policies

2019-11-21 · Siddharth Ghiya, Oluwafemi Azeez, Brendan Miller

Reinforcement learning in a multi agent system is difficult because these systems are inherently non-stationary in nature. In such a case, identifying the type of the opposite agent is crucial and can help us address thi…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Q-Probe: Scaling Image Quality Assessment to High Resolution via Context-Aware Agentic Probing

2026-01-21 · Xiang Li, Xueheng Li, Yu Wang, Xuanhua He 외 arxiv

Reinforcement Learning (RL) has empowered Multimodal Large Language Models (MLLMs) to achieve superior human preference alignment in Image Quality Assessment (IQA). However, existing RL-based IQA models typically rely on…

Image Quality Assessment Reinforcement Learning