Counterfactual contrastive learning: robust representations via causal image synthesis
Contrastive pretraining is well-known to improve downstream task performance and model generalisation, especially in limited label settings. However, it is sensitive to the choice of augmentation pipeline. Positive pairs should preserve semantic information while destroying domain-specific information. Standard augmentation pipelines emulate domain-specific changes with pre-defined photometric transformations, but what if we could simulate realistic domain changes instead? In this work, we show how to utilise recent progress in counterfactual image generation to this effect. We propose CF-SimCLR, a counterfactual contrastive learning approach which leverages approximate counterfactual inference for positive pair creation. Comprehensive evaluation across five datasets, on chest radiography and mammography, demonstrates that CF-SimCLR substantially improves robustness to acquisition shift with higher downstream performance on both in- and out-of-distribution data, particularly for domains which are under-represented during training.
Code (1)
Tasks
Contrastive LearningcounterfactualCounterfactual InferenceImage GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Robust image representations with counterfactual contrastive learning
Contrastive pretraining can substantially increase model generalisation and downstream performance. However, the quality of the learned representations is highly dependent on the data augmentation strategy applied to gen…
Contrastive LearningcounterfactualData AugmentationImage GenerationExplaining Image Classifiers Using Contrastive Counterfactuals in Generative Latent Spaces
Despite their high accuracies, modern complex image classifiers cannot be trusted for sensitive tasks due to their unknown decision-making process and potential biases. Counterfactual explanations are very effective in p…
AttributecounterfactualDecision MakingCausal Image Synthesis of Brain MR in 3D
Clinical decision making requires counterfactual reasoning based on a factual medical image and thus necessitates causal image synthesis. To this end, we present a novel method for modeling the causality between demograp…
counterfactualCounterfactual ReasoningDecision MakingImage GenerationLongitudinal Causal Image Synthesis
Clinical decision-making relies heavily on causal reasoning and longitudinal analysis. For example, for a patient with Alzheimer's disease (AD), how will the brain grey matter atrophy in a year if intervened on the A-bet…
counterfactualImage GenerationMedSteer: Counterfactual Endoscopic Synthesis via Training-Free Activation Steering
Generative diffusion models are increasingly used for medical imaging data augmentation, but text prompting cannot produce causal training data. Re-prompting rerolls the entire generation trajectory, altering anatomy, te…
Data Augmentation