Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators
Neural image classifiers are known to undergo severe performance degradation when exposed to inputs that are sampled from environmental conditions that differ from their training data. Given the recent progress in Text-to-Image (T2I) generation, a natural question is how modern T2I generators can be used to simulate arbitrary interventions over such environmental factors in order to augment training data and improve the robustness of downstream classifiers. We experiment across a diverse collection of benchmarks in single domain generalization (SDG) and reducing reliance on spurious features (RRSF), ablating across key dimensions of T2I generation, including interventional prompting strategies, conditioning mechanisms, and post-hoc filtering. Our extensive empirical findings demonstrate that modern T2I generators like Stable Diffusion can indeed be used as a powerful interventional data augmentation mechanism, outperforming previously state-of-the-art data augmentation techniques regardless of how each dimension is configured.
Code (1)
Tasks
Data AugmentationDomain GeneralizationImage AugmentationRetrievalSimilar Papers 제목 키워드 기반
Beyond Pretty Pictures: Combined Single- and Multi-Image Super-resolution for Sentinel-2 Images
Super-resolution aims to increase the resolution of satellite images by reconstructing high-frequency details, which go beyond na\"ive upsampling. This has particular relevance for Earth observation missions like Sentine…
Earth ObservationImage Super-ResolutionLand Cover ClassificationSuper-ResolutionUnveiling Causalities in SAR ATR: A Causal Interventional Approach for Limited Data
Synthetic aperture radar automatic target recognition (SAR ATR) methods fall short with limited training data. In this letter, we propose a causal interventional ATR method (CIATR) to formulate the problem of limited SAR…
Causal InferenceData AugmentationCompressed Causal Reasoning: Quantization and GraphRAG Effects on Interventional and Counterfactual Accuracy
Causal reasoning in Large Language Models spanning association, intervention, and counterfactual inference is essential for reliable decision making in high stakes settings. As deployment shifts toward edge and resource …
Decision MakingHigh Dimensional Causal Inference with Variational Backdoor Adjustment
Backdoor adjustment is a technique in causal inference for estimating interventional quantities from purely observational data. For example, in medical settings, backdoor adjustment can be used to control for confounding…
Causal InferenceVariational InferenceEfficient adjustment sets for population average treatment effect estimation in non-parametric causal graphical models
The method of covariate adjustment is often used for estimation of population average treatment effects in observational studies. Graphical rules for determining all valid covariate adjustment sets from an assumed causal…
valid