Relative Variational Intrinsic Control
In the absence of external rewards, agents can still learn useful behaviors by identifying and mastering a set of diverse skills within their environment. Existing skill learning methods use mutual information objectives to incentivize each skill to be diverse and distinguishable from the rest. However, if care is not taken to constrain the ways in which the skills are diverse, trivially diverse skill sets can arise. To ensure useful skill diversity, we propose a novel skill learning objective, Relative Variational Intrinsic Control (RVIC), which incentivizes learning skills that are distinguishable in how they change the agent's relationship to its environment. The resulting set of skills tiles the space of affordances available to the agent. We qualitatively analyze skill behaviors on multiple environments and show how RVIC skills are more useful than skills discovered by existing methods when used in hierarchical reinforcement learning.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityHierarchical Reinforcement LearningSimilar Papers 제목 키워드 기반
Variational Intrinsic Control Revisited
In this paper, we revisit variational intrinsic control (VIC), an unsupervised reinforcement learning method for finding the largest set of intrinsic options available to an agent. In the original work by Gregor et al. (…
reinforcement-learningReinforcement Learning (RL)Unsupervised Reinforcement LearningIntrinsic Control of Variational Beliefs in Dynamic Partially-Observed Visual Environments
Humans and animals explore their environment and acquire useful skills even in the absence of clear goals, exhibiting intrinsic motivation. The study of intrinsic motivation in artificial agents is concerned with the fol…
Deep Reinforcement LearningUniversal AI maximizes Variational Empowerment
This paper presents a theoretical framework unifying AIXI -- a model of universal AI -- with variational empowerment as an intrinsic drive for exploration. We build on the existing framework of Self-AIXI -- a universal l…
VASE: Variational Assorted Surprise Exploration for Reinforcement Learning
Exploration in environments with continuous control and sparse rewards remains a key challenge in reinforcement learning (RL). Recently, surprise has been used as an intrinsic reward that encourages systematic and effici…
continuous-controlContinuous ControlEfficient Explorationreinforcement-learning+3Experimental Evidence that Empowerment May Drive Exploration in Sparse-Reward Environments
Reinforcement Learning (RL) is known to be often unsuccessful in environments with sparse extrinsic rewards. A possible countermeasure is to endow RL agents with an intrinsic reward function, or 'intrinsic motivation', w…
Reinforcement Learning (RL)