Towards Understanding Extrapolation: a Causal Lens
Canonical work handling distribution shifts typically necessitates an entire target distribution that lands inside the training distribution. However, practical scenarios often involve only a handful of target samples, potentially lying outside the training support, which requires the capability of extrapolation. In this work, we aim to provide a theoretical understanding of when extrapolation is possible and offer principled methods to achieve it without requiring an on-support target distribution. To this end, we formulate the extrapolation problem with a latent-variable model that embodies the minimal change principle in causal mechanisms. Under this formulation, we cast the extrapolation problem into a latent-variable identification problem. We provide realistic conditions on shift properties and the estimation objectives that lead to identification even when only one off-support target sample is available, tackling the most challenging scenarios. Our theory reveals the intricate interplay between the underlying manifold's smoothness and the shift properties. We showcase how our theoretical results inform the design of practical adaptation algorithms. Through experiments on both synthetic and real-world data, we validate our theoretical findings and their practical implications.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Through a Steerable Lens: Magnifying Neural Network Interpretability via Phase-Based Extrapolation
Understanding the internal representations and decision mechanisms of deep neural networks remains a critical open challenge. While existing interpretability methods often identify influential input regions, they may not…
Motion MagnificationOn Model Extrapolation in Marginal Shapley Values
As the use of complex machine learning models continues to grow, so does the need for reliable explainability methods. One of the most popular methods for model explainability is based on Shapley values. There are two mo…
modelRe-examining Granger Causality from Causal Bayesian Networks Perspective
Characterizing cause-effect relationships in complex systems could be critical to understanding these systems. For many, Granger causality (GC) remains a computational tool of choice to identify causal relations in time …
Causal DiscoveryTime SeriesA Review of Causal Decision Making
To make effective decisions, it is important to have a thorough understanding of the causal relationships among actions, environments, and outcomes. This review aims to surface three crucial aspects of decision-making th…
Decision MakingConcept Modulation Models: A Unified Framework for Identifiability and Extrapolation
Reliable generalization in conditional latent variable models requires understanding both identifiability and extrapolation: how observed variation across attributes determines latent structure, and how that structure de…
Representation Learning