Identifying Spurious Correlations for Robust Text Classification
The predictions of text classifiers are often driven by spurious correlations -- e.g., the term `Spielberg' correlates with positively reviewed movies, even though the term itself does not semantically convey a positive sentiment. In this paper, we propose a method to distinguish spurious and genuine correlations in text classification. We treat this as a supervised classification problem, using features derived from treatment effect estimators to distinguish spurious correlations from "genuine" ones. Due to the generic nature of these features and their small dimensionality, we find that the approach works well even with limited training examples, and that it is possible to transport the word classifier to new domains. Experiments on four datasets (sentiment classification and toxicity detection) suggest that using this approach to inform feature selection also leads to more robust classification, as measured by improved worst-case accuracy on the samples affected by spurious correlations.
Code (1)
Tasks
Classificationfeature selectionGeneral ClassificationRobust classificationSentiment AnalysisSentiment Classificationtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Identifying Spurious Correlations and Correcting them with an Explanation-based Learning
Identifying spurious correlations learned by a trained model is at the core of refining a trained model and building a trustworthy model. We present a simple method to identify spurious correlations that have been learne…
Classificationimage-classificationImage ClassificationExplore Spurious Correlations at the Concept Level in Language Models for Text Classification
Language models (LMs) have achieved notable success in numerous NLP tasks, employing both fine-tuning and in-context learning (ICL) methods. While language models demonstrate exceptional performance, they face robustness…
counterfactualIn-Context Learningtext-classificationText ClassificationIdentifying and Disentangling Spurious Features in Pretrained Image Representations
Neural networks employ spurious correlations in their predictions, resulting in decreased performance when these correlations do not hold. Recent works suggest fixing pretrained representations and training a classificat…
Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language
Spurious correlations are a threat to the trustworthiness of natural language processing systems, motivating research into methods for identifying and eliminating them. However, addressing the problem of spurious correla…
InformativenessVocal Bursts Valence PredictionUnderstanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysis
Recent research has revealed that machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances. For instance, a sentiment classif…
Classificationtext-classificationText Classification