Label Ranking with Partial Abstention based on Thresholded Probabilistic Models
Several machine learning methods allow for abstaining from uncertain predictions. While being common for settings like conventional classification, abstention has been studied much less in learning to rank. We address abstention for the label ranking setting, allowing the learner to declare certain pairs of labels as being incomparable and, thus, to predict partial instead of total orders. In our method, such predictions are produced via thresholding the probabilities of pairwise preferences between labels, as induced by a predicted probability distribution on the set of all rankings. We formally analyze this approach for the Mallows and the Plackett-Luce model, showing that it produces proper partial orders as predictions and characterizing the expressiveness of the induced class of partial orders. These theoretical results are complemented by experiments demonstrating the practical usefulness of the approach.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationLearning-To-RankSimilar Papers 제목 키워드 기반
A comparative analysis of rank aggregation methods for the partial label ranking problem
The \textit{label ranking} problem is a supervised learning scenario in which the learner predicts a \textit{total order} of the class labels for a given input instance. Recently, research has increasingly focused on the…
Reliable Multi-label Classification: Prediction with Partial Abstention
In contrast to conventional (single-label) classification, the setting of multilabel classification (MLC) allows an instance to belong to several classes simultaneously. Thus, instead of selecting a single class label, p…
ClassificationGeneral ClassificationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATION+1Ranking with Abstention
We introduce a novel framework of ranking with abstention, where the learner can abstain from making prediction at some limited cost $c$. We present a extensive theoretical analysis of this framework including a series o…
Who Gets Missed in the Tail? Thresholded Subgroup Underdiagnosis in Long-Tailed Chest X-ray Classification
In chest X-ray (CXR) classification, acceptable ranking performance can still leave rare-positive patients below threshold, especially within subgroups. We study this pre-deployment fairness problem as an audit question:…
Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism
Neural Information Retrieval (NIR) has significantly improved upon heuristic-based Information Retrieval (IR) systems. Yet, failures remain frequent, the models used often being unable to retrieve documents relevant to t…
Information RetrievalRerankingRetrieval