Group conditional validity via multi-group learning
We consider the problem of distribution-free conformal prediction and the criterion of group conditional validity. This criterion is motivated by many practical scenarios including hidden stratification and group fairness. Existing methods achieve such guarantees under either restrictive grouping structure or distributional assumptions, or they are overly-conservative under heteroskedastic noise. We propose a simple reduction to the problem of achieving validity guarantees for individual populations by leveraging algorithms for a problem called multi-group learning. This allows us to port theoretical guarantees from multi-group learning to obtain obtain sample complexity guarantees for conformal prediction. We also provide a new algorithm for multi-group learning for groups with hierarchical structure. Using this algorithm in our reduction leads to improved sample complexity guarantees with a simpler predictor structure.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionFairnessSimilar Papers 제목 키워드 기반
Posterior Conformal Prediction
Conformal prediction is a popular technique for constructing prediction intervals with distribution-free coverage guarantees. The coverage is marginal, meaning it only holds on average over the entire population but not …
Conformal PredictionPredictionPrediction IntervalsSame Same, But Different: Conditional Multi-Task Learning for Demographic-Specific Toxicity Detection
Algorithmic bias often arises as a result of differential subgroup validity, in which predictive relationships vary across groups. For example, in toxic language detection, comments targeting different demographic groups…
Multi-Task LearningLength Optimization in Conformal Prediction
Conditional validity and length efficiency are two crucial aspects of conformal prediction (CP). Conditional validity ensures accurate uncertainty quantification for data subpopulations, while proper length efficiency en…
Conformal PredictionLanguage ModelingLanguage ModellingLarge Language Model+5Subgroup Validity in Machine Learning for Echocardiogram Data
Echocardiogram datasets enable training deep learning models to automate interpretation of cardiac ultrasound, thereby expanding access to accurate readings of diagnostically-useful images. However, the gender, sex, race…
Conformal Classification with Equalized Coverage for Adaptively Selected Groups
This paper introduces a conformal inference method to evaluate uncertainty in classification by generating prediction sets with valid coverage conditional on adaptively chosen features. These features are carefully selec…
Fairnessvalid