Robustifying Sentiment Classification by Maximally Exploiting Few Counterfactuals
For text classification tasks, finetuned language models perform remarkably well. Yet, they tend to rely on spurious patterns in training data, thus limiting their performance on out-of-distribution (OOD) test data. Among recent models aiming to avoid this spurious pattern problem, adding extra counterfactual samples to the training data has proven to be very effective. Yet, counterfactual data generation is costly since it relies on human annotation. Thus, we propose a novel solution that only requires annotation of a small fraction (e.g., 1%) of the original training data, and uses automatic generation of extra counterfactuals in an encoding vector space. We demonstrate the effectiveness of our approach in sentiment classification, using IMDb data for training and other sets for OOD tests (i.e., Amazon, SemEval and Yelp). We achieve noticeable accuracy improvements by adding only 1% manual counterfactuals: +3% compared to adding +100% in-distribution training samples, +1.3% compared to alternate counterfactual approaches.
Code (1)
Tasks
ClassificationcounterfactualSentiment AnalysisSentiment Classificationtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation
While counterfactual data augmentation offers a promising step towards robust generalization in natural language processing, producing a set of counterfactuals that offer valuable inductive bias for models remains a chal…
counterfactualData AugmentationDiversityInductive Bias+4Individualised Counterfactual Examples Using Conformal Prediction Intervals
Counterfactual explanations for black-box models aim to pr ovide insight into an algorithmic decision to its recipient. For a binary classification problem an individual counterfactual details which features might be cha…
Binary ClassificationConformal PredictioncounterfactualData Augmentation+2Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals
Counterfactual Data Augmentation (CDA) is a commonly used technique for improving robustness in natural language classifiers. However, one fundamental challenge is how to discover meaningful counterfactuals and efficient…
counterfactualData AugmentationSentiment AnalysisSentiment ClassificationLess is More: Attention Supervision with Counterfactuals for Text Classification
We aim to leverage human and machine intelligence together for attention supervision. Specifically, we show that human annotation cost can be kept reasonably low, while its quality can be enhanced by machine self-supervi…
ClassificationcounterfactualCounterfactual ReasoningGeneral Classification+3