Flexible and Efficient Long-Range Planning Through Curious Exploration
Identifying algorithms that flexibly and efficiently discover temporally-extended multi-phase plans is an essential step for the advancement of robotics and model-based reinforcement learning. The core problem of long-range planning is finding an efficient way to search through the tree of possible action sequences. Existing non-learned planning solutions from the Task and Motion Planning (TAMP) literature rely on the existence of logical descriptions for the effects and preconditions for actions. This constraint allows TAMP methods to efficiently reduce the tree search problem but limits their ability to generalize to unseen and complex physical environments. In contrast, deep reinforcement learning (DRL) methods use flexible neural-network-based function approximators to discover policies that generalize naturally to unseen circumstances. However, DRL methods struggle to handle the very sparse reward landscapes inherent to long-range multi-step planning situations. Here, we propose the Curious Sample Planner (CSP), which fuses elements of TAMP and DRL by combining a curiosity-guided sampling strategy with imitation learning to accelerate planning. We show that CSP can efficiently discover interesting and complex temporally-extended plans for solving a wide range of physically realistic 3D tasks. In contrast, standard planning and learning methods often fail to solve these tasks at all or do so only with a huge and highly variable number of training samples. We explore the use of a variety of curiosity metrics with CSP and analyze the types of solutions that CSP discovers. Finally, we show that CSP supports task transfer so that the exploration policies learned during experience with one task can help improve efficiency on related tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningImitation LearningModel-based Reinforcement LearningMotion Planningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Task and Motion PlanningSimilar Papers 제목 키워드 기반
Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulation
It has been a long-standing dream to design artificial agents that explore their environment efficiently via intrinsic motivation, similar to how children perform curious free play. Despite recent advances in intrinsical…
Efficient ExplorationObjectReinforcement Learning (RL)Zero-shot GeneralizationEvolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
The ability to perform effective planning is crucial for building an instruction-following agent. When navigating through a new environment, an agent is challenged with (1) connecting the natural language instructions wi…
Decision MakingImitation LearningInstruction FollowingVision and Language NavigationFCRF: Flexible Constructivism Reflection for Long-Horizon Robotic Task Planning with Large Language Models
Autonomous error correction is critical for domestic robots to achieve reliable execution of complex long-horizon tasks. Prior work has explored self-reflection in Large Language Models (LLMs) for task planning error cor…
PEARL: Parallelized Expert-Assisted Reinforcement Learning for Scene Rearrangement Planning
Scene Rearrangement Planning (SRP) is an interior task proposed recently. The previous work defines the action space of this task with handcrafted coarse-grained actions that are inflexible to be used for transforming sc…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)UniPhys: Unified Planner and Controller with Diffusion for Flexible Physics-Based Character Control
Generating natural and physically plausible character motion remains challenging, particularly for long-horizon control with diverse guidance signals. While prior work combines high-level diffusion-based motion planners …
Motion Planning