Learnability with Indirect Supervision Signals
Learning from indirect supervision signals is important in real-world AI applications when, often, gold labels are missing or too costly. In this paper, we develop a unified theoretical framework for multi-class classification when the supervision is provided by a variable that contains nonzero mutual information with the gold label. The nature of this problem is determined by (i) the transition probability from the gold labels to the indirect supervision variables and (ii) the learner's prior knowledge about the transition. Our framework relaxes assumptions made in the literature, and supports learning with unknown, non-invertible and instance-dependent transitions. Our theory introduces a novel concept called \emph{separation}, which characterizes the learnability and generalization bounds. We also demonstrate the application of our framework via concrete novel results in a variety of learning scenarios such as learning with superset annotations and joint supervision signals.
Code (0)
등록된 구현이 없습니다.
Tasks
Generalization BoundsMulti-class ClassificationSimilar Papers 제목 키워드 기반
PABI: A Unified PAC-Bayesian Informativeness Measure for Incidental Supervision Signals
Real-world applications often require making use of {\em a range of incidental supervision signals}. However, we currently lack a principled way to measure the benefit an incidental training dataset can bring, and the co…
Informativenessnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Learning from Indirect Observations
Weakly-supervised learning is a paradigm for alleviating the scarcity of labeled data by leveraging lower-quality but larger-scale supervision signals. While existing work mainly focuses on utilizing a certain type of we…
Weakly-supervised LearningWeak Supervision for Real World Graphs
Node classification in real world graphs often suffers from label scarcity and noise, especially in high stakes domains like human trafficking detection and misinformation monitoring. While direct supervision is limited,…
Contrastive LearningMisinformationNode ClassificationRepresentation LearningCreating Training Sets via Weak Indirect Supervision
Creating labeled training sets has become one of the major roadblocks in machine learning. To address this, recent \emph{Weak Supervision (WS)} frameworks synthesize training labels from multiple potentially noisy superv…
text-classificationText ClassificationDeep Probabilistic Logic: A Unifying Framework for Indirect Supervision
Deep learning has emerged as a versatile tool for a wide range of NLP tasks, due to its superior capacity in representation learning. But its applicability is limited by the reliance on annotated examples, which are diff…
Reading ComprehensionRepresentation Learning