Self-Distilled Disentangled Learning for Counterfactual Prediction
The advancements in disentangled representation learning significantly enhance the accuracy of counterfactual predictions by granting precise control over instrumental variables, confounders, and adjustable variables. An appealing method for achieving the independent separation of these factors is mutual information minimization, a task that presents challenges in numerous machine learning scenarios, especially within high-dimensional spaces. To circumvent this challenge, we propose the Self-Distilled Disentanglement framework, referred to as $SD^2$. Grounded in information theory, it ensures theoretically sound independent disentangled representations without intricate mutual information estimator designs for high-dimensional representations. Our comprehensive experiments, conducted on both synthetic and real-world datasets, confirms the effectiveness of our approach in facilitating counterfactual inference in the presence of both observed and unobserved confounders.
Code (1)
Tasks
counterfactualCounterfactual InferenceDisentanglementPredictionRepresentation LearningSimilar Papers 제목 키워드 기반
An Information Criterion for Controlled Disentanglement of Multimodal Data
Multimodal representation learning seeks to relate and decompose information inherent in multiple modalities. By disentangling modality-specific information from information that is shared across modalities, we can impro…
counterfactualDisentanglementRepresentation LearningSelf-Supervised LearningVisual Disentangled Diffusion Autoencoders: Scalable Counterfactual Generation for Foundation Models
Foundation models, despite their robust zero-shot capabilities, remain vulnerable to spurious correlations and 'Clever Hans' strategies. Existing mitigation methods often rely on unavailable group labels or computational…
Knowledge DistillationCRAFT: Counterfactual Credit Assignment from Free Sibling Rollouts for Self-Distilled Agentic Reinforcement Learning
Self-distilled agentic reinforcement learning augments trajectory-level reward with a token-level distillation loss, using as its teacher the same policy conditioned on privileged context. The prevailing recipe gates thi…
Reinforcement LearningDisentangled and Distilled Encoder for Out-of-Distribution Reasoning with Rademacher Guarantees
Recently, the disentangled latent space of a variational autoencoder (VAE) has been used to reason about multi-label out-of-distribution (OOD) test samples that are derived from different distributions than training samp…
Model CompressionDisentangled Counterfactual Reasoning for Unbiased Sequential Recommendation
Sequential recommender systems have achieved state-of-the-art recommendation performance by modeling the sequential dynamics of user activities. However, in most recommendation scenarios, the popular items comprise the m…
counterfactualCounterfactual ReasoningRecommendation SystemsSequential Recommendation