Increasing Robustness to Spurious Correlations using Forgettable Examples
Neural NLP models tend to rely on spurious correlations between labels and input features to perform their tasks. Minority examples, i.e., examples that contradict the spurious correlations present in the majority of data points, have been shown to increase the out-of-distribution generalization of pre-trained language models. In this paper, we first propose using example forgetting to find minority examples without prior knowledge of the spurious correlations present in the dataset. Forgettable examples are instances either learned and then forgotten during training or never learned. We empirically show how these examples are related to minorities in our training sets. Then, we introduce a new approach to robustify models by fine-tuning our models twice, first on the full training data and second on the minorities only. We obtain substantial improvements in out-of-distribution generalization when applying our approach to the MNLI, QQP, and FEVER datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceNatural Language UnderstandingOut-of-Distribution GeneralizationQQPMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Increasing Robustness to Spurious Correlations using Forgettable Examples
Neural NLP models tend to rely on spurious correlations between labels and input features to perform their tasks. Minority examples, i.e., examples that contradict the spurious correlations present in the majority of dat…
Out-of-Distribution GeneralizationQQPAn Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Recent work has shown that pre-trained language models such as BERT improve robustness to spurious correlations in the dataset. Intrigued by these results, we find that the key to their success is generalization from a s…
DiversityMulti-Task LearningNatural Language InferenceParaphrase IdentificationAre Vision Transformers Robust to Spurious Correlations?
Deep neural networks may be susceptible to learning spurious correlations that hold on average but not in atypical test samples. As with the recent emergence of vision transformer (ViT) models, it remains underexplored h…
Understanding Rare Spurious Correlations in Neural Networks
Neural networks are known to use spurious correlations such as background information for classification. While prior work has looked at spurious correlations that are widespread in the training data, in this work, we in…
Overparameterization hurts worst-group accuracy with spurious correlations
Increasing model capacity well beyond the point of zero training error has been observed to improve average test accuracy. However, such overparameterized models have been recently shown to obtain low worst-group accurac…