Neural Networks for Learning Counterfactual G-Invariances from Single Environments
Despite -- or maybe because of -- their astonishing capacity to fit data, neural networks are believed to have difficulties extrapolating beyond training data distribution. This work shows that, for extrapolations based on finite transformation groups, a model's inability to extrapolate is unrelated to its capacity. Rather, the shortcoming is inherited from a learning hypothesis: Examples not explicitly observed with infinitely many training examples have underspecified outcomes in the learner's model. In order to endow neural networks with the ability to extrapolate over group transformations, we introduce a learning framework counterfactually-guided by the learning hypothesis that any group invariance to (known) transformation groups is mandatory even without evidence, unless the learner deems it inconsistent with the training data. Unlike existing invariance-driven methods for (counterfactual) extrapolations, this framework allows extrapolations from a single environment. Finally, we introduce sequence and image extrapolation tasks that validate our framework and showcase the shortcomings of traditional approaches.
Code (1)
Tasks
counterfactualSimilar Papers 제목 키워드 기반
Neural Network Extrapolations with G-invariances from a Single Environment
Despite —or maybe because of— their astonishing capacity to fit data, neural networks are widely believed to be unable to extrapolate beyond training data distribution. This work shows that, for extrapolations based on t…
counterfactualCounterfactual InferenceTowards Robust Aspect-based Sentiment Analysis through Non-counterfactual Augmentations
While state-of-the-art NLP models have demonstrated excellent performance for aspect based sentiment analysis (ABSA), substantial evidence has been presented on their lack of robustness. This is especially manifested as …
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)counterfactualData Augmentation+1Does Invariant Risk Minimization Capture Invariance?
We show that the Invariant Risk Minimization (IRM) formulation of Arjovsky et al. (2019) can fail to capture "natural" invariances, at least when used in its practical "linear" form, and even on very simple problems whic…
Learning Online Visual Invariances for Novel Objects via Supervised and Self-Supervised Training
Humans can identify objects following various spatial transformations such as scale and viewpoint. This extends to novel objects, after a single presentation at a single pose, sometimes referred to as online invariance. …
Data AugmentationTranslationDeep Neural Networks with Efficient Guaranteed Invariances
We address the problem of improving the performance and in particular the sample complexity of deep neural networks by enforcing and guaranteeing invariances to symmetry transformations rather than learning them from dat…