Variational Temporal Abstraction
We introduce a variational approach to learning and inference of temporally hierarchical structure and representation for sequential data. We propose the Variational Temporal Abstraction (VTA), a hierarchical recurrent state space model that can infer the latent temporal structure and thus perform the stochastic state transition hierarchically. We also propose to apply this model to implement the jumpy-imagination ability in imagination-augmented agent-learning in order to improve the efficiency of the imagination. In experiments, we demonstrate that our proposed method can model 2D and 3D visual sequence datasets with interpretable temporal structure discovery and that its application to jumpy imagination enables more efficient agent-learning in a 3D navigation task.
Code (2)
Similar Papers 제목 키워드 기반
Temporal Difference Variational Auto-Encoder
To act and plan in complex environments, we posit that agents should have a mental simulator of the world with three characteristics: (a) it should build an abstract state representing the condition of the world; (b) it …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Option Transfer and SMDP Abstraction with Successor Features
Abstraction plays an important role in the generalisation of knowledge and skills and is key to sample efficient learning. In this work, we study joint temporal and state abstraction in reinforcement learning, where temp…
Reinforcement Learning (RL)Variational Offline Multi-agent Skill Discovery
Skills are effective temporal abstractions established for sequential decision making, which enable efficient hierarchical learning for long-horizon tasks and facilitate multi-task learning through their transferability.…
Decision MakingMulti-agent Reinforcement LearningMulti-Task LearningSequential Decision Making+1Learning to Abstract with Nonparametric Variational Information Bottleneck
Learned representations at the level of characters, sub-words, words and sentences, have each contributed to advances in understanding different NLP tasks and linguistic phenomena. However, learning textual embeddings is…
Multi-Timescale, Gradient Descent, Temporal Difference Learning with Linear Options
Deliberating on large or continuous state spaces have been long standing challenges in reinforcement learning. Temporal Abstraction have somewhat made this possible, but efficiently planing using temporal abstraction sti…
Reinforcement LearningReinforcement Learning (RL)