Debate as Optimization: Adaptive Conformal Prediction and Diverse Retrieval for Event Extraction

2024-06-18 · Sijia Wang, Lifu Huang

We propose a multi-agent debate as optimization (DAO) system for event extraction, where the primary objective is to iteratively refine the large language models (LLMs) outputs through debating without parameter tuning. In DAO, we introduce two novel modules: the Diverse-RAG (DRAG) module and the Adaptive Conformal Prediction (AdaCP) module. DRAG systematically retrieves supporting information that best fits the debate discussion, while AdaCP enhances the accuracy and reliability of event extraction by effectively rejecting less promising answers. Experimental results demonstrate a significant reduction in the performance gap between supervised approaches and tuning-free LLM-based methods by 18.1% and 17.8% on ACE05 and 17.9% and 15.2% on CASIE for event detection and argument extraction respectively.

📄 PDF Abstract BibTeX arXiv:2406.12197

Code (0)

등록된 구현이 없습니다.

Tasks

Conformal PredictionEvent DetectionEvent Extraction RAG Retrieval

From Debate to Decision: Conformal Social Choice for Safe Multi-Agent Deliberation

2026-04-09 · Mengdie Flora Wang, Haochen Xie, Guanghui Wang, Aijing Gao 외 arxiv

Multi-agent debate improves LLM reasoning, yet agreement among agents is not evidence of correctness. When agents converge on a wrong answer through social reinforcement, consensus-based stopping commits that error to an…

Class Adaptive Conformal Training

2026-01-14 · Badr-Eddine Marani, Julio Silva-Rodriguez, Ismail Ben Ayed, Maria Vakalopoulou 외 arxiv

Deep neural networks have achieved remarkable success across a variety of tasks, yet they often suffer from unreliable probability estimates. As a result, they can be overconfident in their predictions. Conformal Predict…

Text Classification

Adaptive Conformal Prediction for Improving Factuality of Generations by Large Language Models

2026-04-15 · Aleksandr Rubashevskii, Dzianis Piatrashyn, Preslav Nakov, Maxim Panov arxiv

Large language models (LLMs) are prone to generating factually incorrect outputs. Recent work has applied conformal prediction to provide uncertainty estimates and statistical guarantees for the factuality of LLM generat…

Question Answering

SPACR: Single-Pass Adaptive Training of Uncertainty-Aware Conformal Regressors

2026-06-09 · Soundouss Messoudi, Sylvain Rousseau, Sébastien Destercke arxiv

Conformal Prediction (CP) provides robust uncertainty guarantees for predictive models, but is typically applied post hoc, which misaligns model training with the conformal goal of producing efficient (i.e, narrow) inter…

Co-optimization for Adaptive Conformal Prediction

2026-03-02 · Xiaoyi Su, Zhixin Zhou, Rui Luo arxiv

Conformal prediction (CP) provides finite-sample, distribution-free marginal coverage, but standard conformal regression intervals can be inefficient under heteroscedasticity and skewness. In particular, popular construc…