Uninformative Input Features and Counterfactual Invariance: Two Perspectives on Spurious Correlations in Natural Language
The natural language processing community has become increasingly interested in spurious correlations, and in methods for identifying and eliminating them. Gardner et al (2021) argue that due to the compositional nature of language, \emph{all} correlations between labels and individual input features are spurious. This paper analyzes this proposal in the context of a toy example, demonstrating three distinct conditions that can give rise to feature-label correlations through a simple PCFG. Linking the toy example to a structured causal model shows that (1) feature-label correlations can arise even when the label is invariant to interventions on the feature, and (2) feature-label correlations may be absent even when the label \emph{is} sensitive to interventions on the feature. Because input features will be individually correlated with labels except in very rare circumstances, mitigation and stress tests should focus on those correlations that are counterfactually invariant under plausible causal models.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualSimilar Papers 제목 키워드 기반
Counterfactual Invariance to Spurious Correlations in Text Classification
Informally, a 'spurious correlation' is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character; e.g., chang…
Causal InferenceClassificationcounterfactualtext-classification+1Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests
Informally, a 'spurious correlation' is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character; e.g., chang…
Causal Inferencecounterfactualtext-classificationText ClassificationBias Challenges in Counterfactual Data Augmentation
Deep learning models tend not to be out-of-distribution robust primarily due to their reliance on spurious features to solve the task. Counterfactual data augmentations provide a general way of (approximately) achieving …
counterfactualData AugmentationResults on Counterfactual Invariance
In this paper we provide a theoretical analysis of counterfactual invariance. We present a variety of existing definitions, study how they relate to each other and what their graphical implications are. We then turn to t…
counterfactualLearning Counterfactually Invariant Predictors
Notions of counterfactual invariance (CI) have proven essential for predictors that are fair, robust, and generalizable in the real world. We propose graphical criteria that yield a sufficient condition for a predictor t…
counterfactualObject Recognition