Streamlining Conformal Information Retrieval via Score Refinement
Information retrieval (IR) methods, like retrieval augmented generation, are fundamental to modern applications but often lack statistical guarantees. Conformal prediction addresses this by retrieving sets guaranteed to include relevant information, yet existing approaches produce large-sized sets, incurring high computational costs and slow response times. In this work, we introduce a score refinement method that applies a simple monotone transformation to retrieval scores, leading to significantly smaller conformal sets while maintaining their statistical guarantees. Experiments on various BEIR benchmarks validate the effectiveness of our approach in producing compact sets containing relevant information.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionInformation RetrievalRetrievalRetrieval-augmented GenerationSimilar Papers 제목 키워드 기반
Multivariate Standardized Residuals for Conformal Prediction
While split conformal prediction guarantees marginal coverage, approaching the stronger property of conditional coverage is essential for reliable uncertainty quantification. Naive conformal scores, however, suffer from …
CONFLARE: CONFormal LArge language model REtrieval
Retrieval-augmented generation (RAG) frameworks enable large language models (LLMs) to retrieve relevant information from a knowledge base and incorporate it into the context for generating responses. This mitigates hall…
Conformal PredictionLanguage ModelingLanguage ModellingLarge Language Model+5ToolChain-CRC: Conformal Risk Control for Agentic AI Under Retrieval and Tool-Use Drift
Modern AI agents retrieve documents, call tools, check intermediate information, and then produce a final answer or action. This creates a risk-control problem that is not visible from the final answer alone. A final res…
RareCP: Regime-Aware Retrieval for Efficient Conformal Prediction
Recent advances in uncertainty quantification for time series forecasting show that conformal prediction can provide reliable prediction intervals, yet standard conformal methods are often inefficient under temporal depe…
Time Series ForecastingTime Series PredictionResponse Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality
Existing research on Retrieval-Augmented Generation (RAG) primarily focuses on improving overall question-answering accuracy, often overlooking the quality of sub-claims within generated responses. Recent methods that at…
Conformal PredictionQuestion AnsweringRAGRetrieval+1