Multi-Object Representation Learning with Iterative Variational Inference
Human perception is structured around objects which form the basis for our higher-level cognition and impressive systematic generalization abilities. Yet most work on representation learning focuses on feature learning without even considering multiple objects, or treats segmentation as an (often supervised) preprocessing step. Instead, we argue for the importance of learning to segment and represent objects jointly. We demonstrate that, starting from the simple assumption that a scene is composed of multiple entities, it is possible to learn to segment images into interpretable objects with disentangled representations. Our method learns -- without supervision -- to inpaint occluded parts, and extrapolates to scenes with more objects and to unseen objects with novel feature combinations. We also show that, due to the use of iterative variational inference, our system is able to learn multi-modal posteriors for ambiguous inputs and extends naturally to sequences.
Code (6)
Tasks
ObjectRepresentation LearningSystematic GeneralizationUnsupervised Object SegmentationVariational InferenceSimilar Papers 제목 키워드 기반
Efficient Iterative Amortized Inference for Learning Symmetric and Disentangled Multi-Object Representations
Unsupervised multi-object representation learning depends on inductive biases to guide the discovery of object-centric representations that generalize. However, we observe that methods for learning these representations …
DisentanglementObjectRepresentation LearningBrain-like variational inference
Inference in both brains and machines can be formalized by optimizing a shared objective: maximizing the evidence lower bound (ELBO) in machine learning, or minimizing variational free energy (F) in neuroscience (ELBO = …
Variational InferenceOn the Relationship Between Variational Inference and Auto-Associative Memory
In this article, we propose a variational inference formulation of auto-associative memories, allowing us to combine perceptual inference and memory retrieval into the same mathematical framework. In this formulation, th…
RetrievalVariational InferenceA General Method for Amortizing Variational Filtering
We introduce the variational filtering EM algorithm, a simple, general-purpose method for performing variational inference in dynamical latent variable models using information from only past and present variables, i.e. …
Inference OptimizationVariational InferenceEnhancing Unimodal Latent Representations in Multimodal VAEs through Iterative Amortized Inference
Multimodal variational autoencoders (VAEs) aim to capture shared latent representations by integrating information from different data modalities. A significant challenge is accurately inferring representations from any …