Learning and Planning in Complex Action Spaces
Many important real-world problems have action spaces that are high-dimensional, continuous or both, making full enumeration of all possible actions infeasible. Instead, only small subsets of actions can be sampled for the purpose of policy evaluation and improvement. In this paper, we propose a general framework to reason in a principled way about policy evaluation and improvement over such sampled action subsets. This sample-based policy iteration framework can in principle be applied to any reinforcement learning algorithm based upon policy iteration. Concretely, we propose Sampled MuZero, an extension of the MuZero algorithm that is able to learn in domains with arbitrarily complex action spaces by planning over sampled actions. We demonstrate this approach on the classical board game of Go and on two continuous control benchmark domains: DeepMind Control Suite and Real-World RL Suite.
Code (1)
Tasks
continuous-controlContinuous ControlGame of GoMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Model-Based Planning with Discrete and Continuous Actions
Action planning using learned and differentiable forward models of the world is a general approach which has a number of desirable properties, including improved sample complexity over model-free RL methods, reuse of lea…
Compact and Efficient Encodings for Planning in Factored State and Action Spaces with Learned Binarized Neural Network Transition Models
In this paper, we leverage the efficiency of Binarized Neural Networks (BNNs) to learn complex state transition models of planning domains with discretized factored state and action spaces. In order to directly exploit t…
ARCComputational EfficiencyModel-Based Reinforcement Learning with Double Oracle Efficiency in Policy Optimization and Offline Estimation
Reinforcement learning (RL) in large environments often suffers from severe computational bottlenecks, as conventional regret minimization algorithms require repeated, costly calls to planning and statistical estimation …
Reinforcement LearningConceptAgent: LLM-Driven Precondition Grounding and Tree Search for Robust Task Planning and Execution
Robotic planning and execution in open-world environments is a complex problem due to the vast state spaces and high variability of task embodiment. Recent advances in perception algorithms, combined with Large Language …
Common Sense ReasoningLogical FallaciesTask PlanningFrom semantics to execution: Integrating action planning with reinforcement learning for robotic causal problem-solving
Reinforcement learning is an appropriate and successful method to robustly perform low-level robot control under noisy conditions. Symbolic action planning is useful to resolve causal dependencies and to break a causally…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)