Conformal Prediction for Natural Language Processing: A Survey
The rapid proliferation of large language models and natural language processing (NLP) applications creates a crucial need for uncertainty quantification to mitigate risks such as hallucinations and to enhance decision-making reliability in critical applications. Conformal prediction is emerging as a theoretically sound and practically useful framework, combining flexibility with strong statistical guarantees. Its model-agnostic and distribution-free nature makes it particularly promising to address the current shortcomings of NLP systems that stem from the absence of uncertainty quantification. This paper provides a comprehensive survey of conformal prediction techniques, their guarantees, and existing applications in NLP, pointing to directions for future research and open challenges.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionDecision MakingPredictionSurveyUncertainty QuantificationSimilar Papers 제목 키워드 기반
Conformal Risk Control
We extend conformal prediction to control the expected value of any monotone loss function. The algorithm generalizes split conformal prediction together with its coverage guarantee. Like conformal prediction, the confor…
Conformal PredictionPredictionDesign-based conformal prediction
Conformal prediction is an assumption-lean approach to generating distribution-free prediction intervals or sets, for nearly arbitrary predictive models, with guaranteed finite-sample coverage. Conformal methods are an a…
Conformal PredictionPredictionPrediction IntervalsSurveyFew-shot Conformal Prediction with Auxiliary Tasks
We develop a novel approach to conformal prediction when the target task has limited data available for training. Conformal prediction identifies a small set of promising output candidates in place of a single prediction…
Computational chemistryConformal PredictionDrug DiscoveryMeta-Learning+1Conformal Autoregressive Generation: Beam Search with Coverage Guarantees
We introduce two new extensions to the beam search algorithm based on conformal predictions (CP) to produce sets of sequences with theoretical coverage guarantees. The first method is very simple and proposes dynamically…
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
Black-box machine learning models are now routinely used in high-risk settings, like medical diagnostics, which demand uncertainty quantification to avoid consequential model failures. Conformal prediction is a user-frie…
Conformal PredictionDeep Reinforcement LearningOut-of-Distribution DetectionPrediction+2