Learning Structured Latent Factors from Dependent Data:A Generative Model Framework from Information-Theoretic Perspective
Learning controllable and generalizable representation of multivariate data with desired structural properties remains a fundamental problem in machine learning. In this paper, we present a novel framework for learning generative models with various underlying structures in the latent space. We represent the inductive bias in the form of mask variables to model the dependency structure in the graphical model and extend the theory of multivariate information bottleneck to enforce it. Our model provides a principled approach to learn a set of semantically meaningful latent factors that reflect various types of desired structures like capturing correlation or encoding invariance, while also offering the flexibility to automatically estimate the dependency structure from data. We show that our framework unifies many existing generative models and can be applied to a variety of tasks including multi-modal data modeling, algorithmic fairness, and invariant risk minimization.
Code (0)
등록된 구현이 없습니다.
Tasks
FairnessInductive BiasSimilar Papers 제목 키워드 기반
Structured Recognition for Generative Models with Explaining Away
A key goal of unsupervised learning is to go beyond density estimation and sample generation to reveal the structure inherent within observed data. Such structure can be expressed in the pattern of interactions between e…
Density EstimationHippocampusTime Series AnalysisVariational InferenceExplicitly disentangling image content from translation and rotation with spatial-VAE
Given an image dataset, we are often interested in finding data generative factors that encode semantic content independently from pose variables such as rotation and translation. However, current disentanglement approac…
DisentanglementTranslationLearning to Manipulate Individual Objects in an Image
We describe a method to train a generative model with latent factors that are (approximately) independent and localized. This means that perturbing the latent variables affects only local regions of the synthesized image…
DisentanglementDisentangling and Learning Robust Representations with Natural Clustering
Learning representations that disentangle the underlying factors of variability in data is an intuitive way to achieve generalization in deep models. In this work, we address the scenario where generative factors present…
ClusteringMulti-View Data Generation Without View Supervision
The development of high-dimensional generative models has recently gained a great surge of interest with the introduction of variational auto-encoders and generative adversarial neural networks. Different variants have b…