Learning to Control Visual Abstractions for Structured Exploration in Deep Reinforcement Learning
Exploration in environments with sparse rewards is a key challenge for reinforcement learning. How do we design agents with generic inductive biases so that they can explore in a consistent manner instead of just using local exploration schemes like epsilon-greedy? We propose an unsupervised reinforcement learning agent which learns a discrete pixel grouping model that preserves spatial geometry of the sensors and implicitly of the environment as well. We use this representation to derive geometric intrinsic reward functions, like centroid coordinates and area, and learn policies to control each one of them with off-policy learning. These policies form a basis set of behaviors (options) which allows us explore in a consistent way and use them in a hierarchical reinforcement learning setup to solve for extrinsically defined rewards. We show that our approach can scale to a variety of domains with competitive performance, including navigation in 3D environments and Atari games with sparse rewards.
Code (0)
등록된 구현이 없습니다.
Tasks
Atari GamesDeep Reinforcement LearningHierarchical Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Unsupervised Reinforcement LearningSimilar Papers 제목 키워드 기반
State Abstractions for Lifelong Reinforcement Learning
In lifelong reinforcement learning, agents must effectively transfer knowledge across tasks while simultaneously addressing exploration, credit assignment, and generalization. State abstraction can help overcome the…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)From Curiosity to Competence: How World Models Interact with the Dynamics of Exploration
What drives an agent to explore the world while also maintaining control over the environment? From a child at play to scientists in the lab, intelligent agents must balance curiosity (the drive to seek knowledge) with c…
reinforcement-learningReinforcement LearningRepresentation LearningLearning Task Informed Abstractions
Current model-based reinforcement learning methods struggle when operating from complex visual scenes due to their inability to prioritize task-relevant features. To mitigate this problem, we propose learning Task Inform…
Model-based Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Learning Task Informed Abstractions
Current model-based reinforcement learning methods struggle when operating from complex visual scenes due to their inability to prioritize task-relevant features. To mitigate this problem, we propose learning Task Inform…
Model-based Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Incremental Hierarchical Reinforcement Learning with Multitask LMDPs
Exploration is a well known challenge in Reinforcement Learning. One principled way of overcoming this challenge is to find a hierarchical abstraction of the base problem and explore at these higher levels, rather than i…
Hierarchical Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)