Conformal Classification with Equalized Coverage for Adaptively Selected Groups
This paper introduces a conformal inference method to evaluate uncertainty in classification by generating prediction sets with valid coverage conditional on adaptively chosen features. These features are carefully selected to reflect potential model limitations or biases. This can be useful to find a practical compromise between efficiency -- by providing informative predictions -- and algorithmic fairness -- by ensuring equalized coverage for the most sensitive groups. We demonstrate the validity and effectiveness of this method on simulated and real data sets.
Code (1)
Tasks
FairnessvalidSimilar Papers 제목 키워드 기반
Fair Conformal Classification via Learning Representation-Based Groups
Conformal prediction methods provide statistically rigorous marginal coverage guarantees for machine learning models, but such guarantees fail to account for algorithmic biases, thereby undermining fairness and trust. Th…
On the Burden of Achieving Fairness in Conformal Prediction
Conformal prediction is often calibrated with a single pooled threshold, but this can hide cross-group heterogeneity in score distributions and distort group-wise coverage. We study this phenomenon through the population…
Conformal Prediction Sets Can Cause Disparate Impact
Although conformal prediction is a promising method for quantifying the uncertainty of machine learning models, the prediction sets it outputs are not inherently actionable. Many applications require a single output to a…
Conformal PredictionFairnessPredictionConformalizing Machine Translation Evaluation
Several uncertainty estimation methods have been recently proposed for machine translation evaluation. While these methods can provide a useful indication of when not to trust model predictions, we show in this paper tha…
Conformal PredictionMachine TranslationPredictionTranslationSelecting informative conformal prediction sets with false coverage rate control
In supervised learning, including regression and classification, conformal methods provide prediction sets for the outcome/label with finite sample coverage for any machine learning predictor. We consider here the case w…
Conformal PredictionPredictionPrediction Intervalsregression