Active World Model Learning with Progress Curiosity
World models are self-supervised predictive models of how the world evolves. Humans learn world models by curiously exploring their environment, in the process acquiring compact abstractions of high bandwidth sensory inputs, the ability to plan across long temporal horizons, and an understanding of the behavioral patterns of other agents. In this work, we study how to design such a curiosity-driven Active World Model Learning (AWML) system. To do so, we construct a curious agent building world models while visually exploring a 3D physical environment rich with distillations of representative real-world agents. We propose an AWML system driven by $\gamma$-Progress: a scalable and effective learning progress-based curiosity signal. We show that $\gamma$-Progress naturally gives rise to an exploration policy that directs attention to complex but learnable dynamics in a balanced manner, thus overcoming the "white noise problem". As a result, our $\gamma$-Progress-driven controller achieves significantly higher AWML performance than baseline controllers equipped with state-of-the-art exploration strategies such as Random Network Distillation and Model Disagreement.
Code (0)
등록된 구현이 없습니다.
Tasks
modelSimilar Papers 제목 키워드 기반
Active World Model Learning in Agent-rich Environments with Progress Curiosity
World models are a family of predictive models that solve self-supervised problems on how the world evolves. Humans learn world models by curiously exploring their environment, in the process acquiring compact abstractio…
An Adversarial Objective for Scalable Exploration
Model-based curiosity combines active learning approaches to optimal sampling with the information gain based incentives for exploration presented in the curiosity literature. Existing model-based curiosity methods look …
Active LearningPredictionCuriosity as filling, compressing, and reconfiguring knowledge networks
Due to the significant role that curiosity plays in our lives, several theoretical constructs, such as the information gap theory and compression progress theory, have sought to explain how we engage in its practice. Acc…
Active Player Modelling
We argue for the use of active learning methods for player modelling. In active learning, the learning algorithm chooses where to sample the search space so as to optimise learning progress. We hypothesise that player mo…
Active LearningFormDynCur-Geo: Dynamic Curiosity Reward Shaping for Multimodal Active Geo-Localization
Active geo-localization enables low-altitude UAVs to search for specified targets from limited local aerial observations, supporting time-sensitive applications such as search and rescue and emergency inspection. However…