Trading Coverage for Precision: Conformal Prediction with Limited False Discoveries
In this paper, we develop a new approach to conformal prediction in which we aim to output a precise set of promising prediction candidates that is guaranteed to contain a limited number of incorrect answers. Standard conformal prediction provides the ability to adapt to model uncertainty by constructing a calibrated candidate set in place of a single prediction, with guarantees that the set contains the correct answer with high probability. In order to obey this coverage property, however, conformal sets can often become inundated with noisy candidates---which can render them unhelpful in practice. This is particularly relevant to large-scale settings where the cost (monetary or otherwise) of false positives is substantial, such as for in-silico screening for drug discovery, where any positively identified molecular compound is then manufactured and tested. We propose to trade coverage for precision by enforcing that the presence of incorrect candidates in the predicted conformal sets (i.e., the total number of false discoveries) is bounded according to a user-specified tolerance. Subject to this constraint, our algorithm then optimizes for a generalized notion of set coverage (i.e., the true discovery rate) that allows for any number of true answers for a given query (including zero). We demonstrate the effectiveness of this approach across a number of classification tasks in natural language processing, computer vision, and computational chemistry.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational chemistryConformal PredictionDrug DiscoveryPredictionSimilar Papers 제목 키워드 기반
Post-selection Inference for Conformal Prediction: Trading off Coverage for Precision
Conformal inference has played a pivotal role in providing uncertainty quantification for black-box ML prediction algorithms with finite sample guarantees. Traditionally, conformal prediction inference requires a data-in…
Binary ClassificationConformal PredictionPredictionUncertainty QuantificationAccelerating Conformal Prediction via Approximate Leave-One-Out
While conformal prediction provides a general framework for uncertainty quantification in predictive inference, its application is often limited by computational cost. Recent methods, including Jackknife+ and Jackknife-m…
Conformal Prediction Sets with Limited False Positives
We develop a new approach to multi-label conformal prediction in which we aim to output a precise set of promising prediction candidates with a bounded number of incorrect answers. Standard conformal prediction provides …
Computational chemistryConformal PredictionPredictionConformal Prediction for Electricity Price Forecasting in the Day-Ahead and Real-Time Balancing Market
The integration of renewable energy into electricity markets poses significant challenges to price stability and increases the complexity of market operations. Accurate and reliable electricity price forecasting is cruci…
Conformal PredictionDecision Makingenergy tradingPrediction+2Conformalised imprecise inference for robust extrapolation under limited data
Recent advances in uncertainty quantification increasingly emphasise the distinction between aleatory and epistemic uncertainty in machine learning, motivating the need for more unified frameworks. However, despite much …