Generalization Bounds for Causal Regression: Insights, Guarantees and Sensitivity Analysis
Many algorithms have been recently proposed for causal machine learning. Yet, there is little to no theory on their quality, especially considering finite samples. In this work, we propose a theory based on generalization bounds that provides such guarantees. By introducing a novel change-of-measure inequality, we are able to tightly bound the model loss in terms of the deviation of the treatment propensities over the population, which we show can be empirically limited. Our theory is fully rigorous and holds even in the face of hidden confounding and violations of positivity. We demonstrate our bounds on semi-synthetic and real data, showcasing their remarkable tightness and practical utility.
Code (1)
Tasks
Generalization BoundsregressionSensitivitySimilar Papers 제목 키워드 기반
Operator-Based Generalization Bound for Deep Learning: Insights on Multi-Task Learning
This paper presents novel generalization bounds for vector-valued neural networks and deep kernel methods, focusing on multi-task learning through an operator-theoretic framework. Our key development lies in strategicall…
Multi-Task LearningUnderstanding the Capabilities and Limitations of Weak-to-Strong Generalization
Weak-to-strong generalization, where weakly supervised strong models outperform their weaker teachers, offers a promising approach to aligning superhuman models with human values. To deepen the understanding of this appr…
Expectation Error Bounds for Transfer Learning in Linear Regression and Linear Neural Networks
In transfer learning, the learner leverages auxiliary data to improve generalization on a main task. However, the precise theoretical understanding of when and how auxiliary data help remains incomplete. We provide new i…
Transfer LearningKernel Regression in Structured Non-IID Settings: Theory and Implications for Denoising Score Learning
Kernel ridge regression (KRR) is a foundational tool in machine learning, with recent work emphasizing its connections to neural networks. However, existing theory primarily addresses the i.i.d. setting, while real-world…
Causal Forecasting:Generalization Bounds for Autoregressive Models
Despite the increasing relevance of forecasting methods, causal implications of these algorithms remain largely unexplored. This is concerning considering that, even under simplifying assumptions such as causal sufficien…
Learning TheoryTime SeriesTime Series Analysis