Growing Action Spaces
In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space. Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data, value estimates, and state representations from restricted action spaces to the full task. We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.
Code (1)
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)StarcraftSimilar Papers 제목 키워드 기반
Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution
Recent reinforcement learning approaches have shown surprisingly strong capabilities of bang-bang policies for solving continuous control benchmarks. The underlying coarse action space discretizations often yield favoura…
continuous-controlContinuous ControlQ-LearningRiemannian L-systems: Modeling growing forms in curved spaces
In the past 50 years, the formalism of L-systems has been successfully used and developed to model the growth of filamentous and branching biological forms. These simulations take place in classical 2-D or 3-D Euclidean …
POMDP-based Object Search with Growing State Space and Hybrid Action Domain
Efficiently locating target objects in complex indoor environments with diverse furniture, such as shelves, tables, and beds, is a significant challenge for mobile robots. This difficulty arises from factors like localiz…
CLAS: Coordinating Multi-Robot Manipulation with Central Latent Action Spaces
Multi-robot manipulation tasks involve various control entities that can be separated into dynamically independent parts. A typical example of such real-world tasks is dual-arm manipulation. Learning to naively solve suc…
Robot ManipulationEfficient Planning in a Compact Latent Action Space
Planning-based reinforcement learning has shown strong performance in tasks in discrete and low-dimensional continuous action spaces. However, planning usually brings significant computational overhead for decision-makin…
continuous-controlContinuous ControlDecision MakingDecoder+1