DISENTANGLED STATE SPACE MODELS: UNSUPERVISED LEARNING OF DYNAMICS ACROSS HETEROGENEOUS ENVIRONMENTS
Sequential data often originates from diverse environments. Across them exist both shared regularities and environment specifics. To learn robust cross-environment descriptions of sequences we introduce disentangled state space models (DSSM). In the latent space of DSSM environment-invariant state dynamics is explicitly disentangled from environment-specific information governing that dynamics. We empirically show that such separation enables robust prediction, sequence manipulation and environment characterization. We also propose an unsupervised VAE-based training procedure to learn DSSM as Bayesian filters. In our experiments, we demonstrate state-of-the-art performance in controlled generation and prediction of bouncing ball video sequences across varying gravitational influences.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionState Space ModelsSimilar Papers 제목 키워드 기반
Disentangled State Space Representations
Sequential data often originates from diverse domains across which statistical regularities and domain specifics exist. To specifically learn cross-domain sequence representations, we introduce disentangled state space m…
regressionState Space ModelsTransfer LearningDisentangled Predictive Representation for Meta-Reinforcement Learning
A major challenge in reinforcement learning is the design of agents that are able to generalize across tasks that share common dynamics. A viable solution is meta-reinforcement learning, which identifies common struc…
Meta Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Experts
Disentangled representations support a range of downstream tasks including causal reasoning, generative modeling, and fair machine learning. Unfortunately, disentanglement has been shown to be impossible without the inco…
DecoderDisentanglementInductive BiasMixture-of-ExpertsA Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
This paper takes a step towards temporal reasoning in a dynamically changing video, not in the pixel space that constitutes its frames, but in a latent space that describes the non-linear dynamics of the objects in its w…
ImputationExploring Behavior-Relevant and Disentangled Neural Dynamics with Generative Diffusion Models
Understanding the neural basis of behavior is a fundamental goal in neuroscience. Current research in large-scale neuro-behavioral data analysis often relies on decoding models, which quantify behavioral information in n…
Disentanglement