Conjunction Subspaces Test for Conformal and Selective Classification
In this paper, we present a new classifier, which integrates significance testing results over different random subspaces to yield consensus p-values for quantifying the uncertainty of classification decision. The null hypothesis is that the test sample has no association with the target class on a randomly chosen subspace, and hence the classification problem can be formulated as a problem of testing for the conjunction of hypotheses. The proposed classifier can be easily deployed for the purpose of conformal prediction and selective classification with reject and refine options by simply thresholding the consensus p-values. The theoretical analysis on the generalization error bound of the proposed classifier is provided and empirical studies on real data sets are conducted as well to demonstrate its effectiveness.
Code (1)
Tasks
ClassificationConformal PredictionSimilar Papers 제목 키워드 기반
Selective Conformal Risk Control
Reliable uncertainty quantification is essential for deploying machine learning systems in high-stakes domains. Conformal prediction provides distribution-free coverage guarantees but often produces overly large predicti…
Online Selective Conformal Prediction: Errors and Solutions
In online selective conformal inference, data arrives sequentially, and prediction intervals are constructed only when an online selection rule is met. Since online selections may break the exchangeability between the se…
Conformal PredictionPredictionPrediction IntervalsvalidOnline Selective Conformal Prediction with Asymmetric Rules: A Permutation Test Approach
Selective conformal prediction aims to construct prediction sets with valid coverage for a test unit conditional on it being selected by a data-driven mechanism. While existing methods in the offline setting handle any s…
Drug DiscoverySConU: Selective Conformal Uncertainty in Large Language Models
As large language models are increasingly utilized in real-world applications, guarantees of task-specific metrics are essential for their reliable deployment. Previous studies have introduced various criteria of conform…
Conformal PredictionQuestion AnsweringApproximate Conditional Coverage & Calibration via Neural Model Approximations
A typical desideratum for quantifying the uncertainty from a classification model as a prediction set is class-conditional singleton set calibration. That is, such sets should map to the output of well-calibrated selecti…
ClassificationDocument ClassificationFact VerificationGrammatical Error Detection+3