Subequivariant Graph Reinforcement Learning in 3D Environments
Learning a shared policy that guides the locomotion of different agents is of core interest in Reinforcement Learning (RL), which leads to the study of morphology-agnostic RL. However, existing benchmarks are highly restrictive in the choice of starting point and target point, constraining the movement of the agents within 2D space. In this work, we propose a novel setup for morphology-agnostic RL, dubbed Subequivariant Graph RL in 3D environments (3D-SGRL). Specifically, we first introduce a new set of more practical yet challenging benchmarks in 3D space that allows the agent to have full Degree-of-Freedoms to explore in arbitrary directions starting from arbitrary configurations. Moreover, to optimize the policy over the enlarged state-action space, we propose to inject geometric symmetry, i.e., subequivariance, into the modeling of the policy and Q-function such that the policy can generalize to all directions, improving exploration efficiency. This goal is achieved by a novel SubEquivariant Transformer (SET) that permits expressive message exchange. Finally, we evaluate the proposed method on the proposed benchmarks, where our method consistently and significantly outperforms existing approaches on single-task, multi-task, and zero-shot generalization scenarios. Extensive ablations are also conducted to verify our design. Code and videos are available on our project page: https://alpc91.github.io/SGRL/.
Code (1)
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Zero-shot GeneralizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Subequivariant Reinforcement Learning in 3D Multi-Entity Physical Environments
Learning policies for multi-entity systems in 3D environments is far more complicated against single-entity scenarios, due to the exponential expansion of the global state space as the number of entities increases. One p…
reinforcement-learningReinforcement LearningSubequivariant Reinforcement Learning Framework for Coordinated Motion Control
Effective coordination is crucial for motion control with reinforcement learning, especially as the complexity of agents and their motions increases. However, many existing methods struggle to account for the intricate d…
reinforcement-learningReinforcement LearningLearning Physical Dynamics with Subequivariant Graph Neural Networks
Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics. However, they still encounter several challenges: 1) Physical laws abide by symmetry, which is a vital inductive bias accounting …
Graph Neural NetworkInductive BiasGraph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments
Most reinforcement learning approaches used in behavior generation utilize vectorial information as input. However, this requires the network to have a pre-defined input-size -- in semantic environments this means assumi…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Foresight of Graph Reinforcement Learning Latent Permutations Learnt by Gumbel Sinkhorn Network
Vital importance has necessity to be attached to cooperation in multi-agent environments, as a result of which some reinforcement learning algorithms combined with graph neural networks have been proposed to understand t…
Graph Attentionreinforcement-learningReinforcement LearningReinforcement Learning (RL)