Query-Adaptive Predictive Inference with Partial Labels
The cost and scarcity of fully supervised labels in statistical machine learning encourage using partially labeled data for model validation as a cheaper and more accessible alternative. Effectively collecting and leveraging weakly supervised data for large-space structured prediction tasks thus becomes an important part of an end-to-end learning system. We propose a new computationally-friendly methodology to construct predictive sets using only partially labeled data on top of black-box predictive models. To do so, we introduce "probe" functions as a way to describe weakly supervised instances and define a false discovery proportion-type loss, both of which seamlessly adapt to partial supervision and structured prediction -- ranking, matching, segmentation, multilabel or multiclass classification. Our experiments highlight the validity of our predictive set construction as well as the attractiveness of a more flexible user-dependent loss framework.
Code (0)
등록된 구현이 없습니다.
Tasks
Structured PredictionSimilar Papers 제목 키워드 기반
Predictive Modeling with Delayed Information: a Case Study in E-commerce Transaction Fraud Control
In Business Intelligence, accurate predictive modeling is the key for providing adaptive decisions. We studied predictive modeling problems in this research which was motivated by real-world cases that Microsoft data sci…
Decision MakingExploiting Counter-Examples for Active Learning with Partial labels
This paper studies a new problem, \emph{active learning with partial labels} (ALPL). In this setting, an oracle annotates the query samples with partial labels, relaxing the oracle from the demanding accurate labeling pr…
Active LearningInstance-Dependent Partial Label Learning
Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true. Most existing PLL approaches assume tha…
Partial Label LearningWeakly-supervised LearningPartial Scene Text Retrieval
The task of partial scene text retrieval involves localizing and searching for text instances that are the same or similar to a given query text from an image gallery. However, existing methods can only handle text-line …
Multiple Instance LearningRetrievalText RetrievalPredictive Inference with Weak Supervision
The expense of acquiring labels in large-scale statistical machine learning makes partially and weakly-labeled data attractive, though it is not always apparent how to leverage such data for model fitting or validation. …
Conformal PredictionStructured Predictionvalid