BeBold: Exploration Beyond the Boundary of Explored Regions
Efficient exploration under sparse rewards remains a key challenge in deep reinforcement learning. To guide exploration, previous work makes extensive use of intrinsic reward (IR). There are many heuristics for IR, including visitation counts, curiosity, and state-difference. In this paper, we analyze the pros and cons of each method and propose the regulated difference of inverse visitation counts as a simple but effective criterion for IR. The criterion helps the agent explore Beyond the Boundary of explored regions and mitigates common issues in count-based methods, such as short-sightedness and detachment. The resulting method, BeBold, solves the 12 most challenging procedurally-generated tasks in MiniGrid with just 120M environment steps, without any curriculum learning. In comparison, the previous SoTA only solves 50% of the tasks. BeBold also achieves SoTA on multiple tasks in NetHack, a popular rogue-like game that contains more challenging procedurally-generated environments.
Code (2)
Tasks
Deep Reinforcement LearningEfficient ExplorationNetHackSimilar Papers 제목 키워드 기반
Learning-Driven Exploration for Reinforcement Learning
Effective and intelligent exploration has been an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies such as $\epsilon$-greedy exploration or add…
Efficient ExplorationFPS Gamesreinforcement-learningReinforcement Learning+1Systematic design space exploration by learning the explored space using Machine Learning
Current practice in parameter space exploration in euclidean space is dominated by randomized sampling or design of experiment methods. The biggest issue with these methods is not keeping track of what part of parameter …
Expedition & Expansion: Leveraging Semantic Representations for Goal-Directed Exploration in Continuous Cellular Automata
Discovering diverse visual patterns in continuous cellular automata (CA) is challenging due to the vastness and redundancy of high-dimensional behavioral spaces. Traditional exploration methods like Novelty Search (NS) e…
Deep exploration by novelty-pursuit with maximum state entropy
Efficient exploration is essential to reinforcement learning in huge state space. Recent approaches to address this issue include the intrinsically motivated goal exploration process (IMGEP) and the maximum state entropy…
Efficient ExplorationMuJoCoBeyond $χ^2$ Difference: Learning Optimal Metric for Boundary Detection
This letter focuses on solving the challenging problem of detecting natural image boundaries. A boundary usually refers to the border between two regions with different semantic meanings. Therefore, a measurement of diss…
Boundary Detection