paper-with-me

Papers

Selective Prediction from Agreement: A Lipschitz-Consistent Version Space Approach

2026-05-04 · Mohamadsadegh Khosravani arxiv

We consider selective classification with abstention in the fixed-pool (or transductive) setting, where the unlabeled pool is given beforehand and only a subset of points can be queried for labels. Our main insight is to view selective prediction through agreement: given queried labels and Lipschitz margin constraints in an embedding space, the version space of Lipschitz-consistent classification heads is well defined. We obtain upper and lower Lipschitz margin bounds that define, for each pool point, a set of certified valid labels containing the prediction of every head in the version space. The model therefore predicts only when the label is forced (i.e., all consistent heads agree), and abstains otherwise. We also propose a monotone submodular geometric proxy for budgeted querying, and show that a greedy algorithm retains the standard approximation factor.

📄 PDF Abstract BibTeX arXiv:2605.02611

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Selective Ensembles for Consistent Predictions

2021-11-16 · ICLR 2022 4 · Emily Black, Klas Leino, Matt Fredrikson

Recent work has shown that models trained to the same objective, and which achieve similar measures of accuracy on consistent test data, may nonetheless behave very differently on individual predictions. This inconsisten…

Medical Diagnosis

The Relationship Between Agnostic Selective Classification Active Learning and the Disagreement Coefficient

2017-03-19 · Roei Gelbhart, Ran El-Yaniv

A selective classifier (f,g) comprises a classification function f and a binary selection function g, which determines if the classifier abstains from prediction, or uses f to predict. The classifier is called pointwise-…

Active LearningGeneral Classification

Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?

2024-08-26 · Urja Khurana, Eric Nalisnick, Antske Fokkens, Swabha Swayamdipta

Subjective tasks in NLP have been mostly relegated to objective standards, where the gold label is decided by taking the majority vote. This obfuscates annotator disagreement and the inherent uncertainty of the label. We…

Decision MakingHate Speech DetectionNatural Language Inference

Agnostic Selective Classification

2011-12-01 · NeurIPS 2011 12 · Yair Wiener, Ran El-Yaniv

For a learning problem whose associated excess loss class is $(\beta,B)$-Bernstein, we show that it is theoretically possible to track the same classification performance of the best (unknown) hypothesis in our class, p…

ClassificationGeneral Classification

When Does Demographic Information Help? Data and Modeling Regimes for Perspective-Aware Hate Speech Detection

2026-05-26 · Weibin Cai, Reza Zafarani arxiv

Demographic information is often used to model annotator perspectives in subjective tasks such as hate speech detection, but its benefit is inconsistent: it improves performance in some settings and behaves as noise in o…

Hate Speech Detection