Training Classifiers with Natural Language Explanations
Training accurate classifiers requires many labels, but each label provides only limited information (one bit for binary classification). In this work, we propose BabbleLabble, a framework for training classifiers in which an annotator provides a natural language explanation for each labeling decision. A semantic parser converts these explanations into programmatic labeling functions that generate noisy labels for an arbitrary amount of unlabeled data, which is used to train a classifier. On three relation extraction tasks, we find that users are able to train classifiers with comparable F1 scores from 5-100$\times$ faster by providing explanations instead of just labels. Furthermore, given the inherent imperfection of labeling functions, we find that a simple rule-based semantic parser suffices.
Code (2)
Tasks
Binary ClassificationGeneral ClassificationRelation ExtractionSimilar Papers 제목 키워드 기반
CLUES: A Benchmark for Learning Classifiers using Natural Language Explanations
Supervised learning has traditionally focused on inductive learning by observing labeled examples of a task. In contrast, humans have the ability to learn new concepts from language. Here, we explore training zero-shot c…
Inductive LearningDISCERN: Decoding Systematic Errors in Natural Language for Text Classifiers
Despite their high predictive accuracies, current machine learning systems often exhibit systematic biases stemming from annotation artifacts or insufficient support for certain classes in the dataset. Recent work propos…
Active Learningtext-classificationText ClassificationZero-shot Learning of Classifiers from Natural Language Quantification
Humans can efficiently learn new concepts using language. We present a framework through which a set of explanations of a concept can be used to learn a classifier without access to any labeled examples. We use semantic …
Semantic ParsingZero-Shot LearningGenerating Token-Level Explanations for Natural Language Inference
The task of Natural Language Inference (NLI) is widely modeled as supervised sentence pair classification. While there has been a lot of work recently on generating explanations of the predictions of classifiers on a sin…
Multiple Instance LearningNatural Language InferenceSentenceSentence-Pair ClassificationLLMs are One-Shot URL Classifiers and Explainers
Malicious URL classification represents a crucial aspect of cyber security. Although existing work comprises numerous machine learning and deep learning-based URL classification models, most suffer from generalisation an…
Domain AdaptationInformativenessOne-Shot Learning