MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models
Developing versatile quadruped robots that can smoothly perform various actions and tasks in real-world environments remains a significant challenge. This paper introduces a novel vision-language-action (VLA) model, mixture of robotic experts (MoRE), for quadruped robots that aim to introduce reinforcement learning (RL) for fine-tuning large-scale VLA models with a large amount of mixed-quality data. MoRE integrates multiple low-rank adaptation modules as distinct experts within a dense multi-modal large language model (MLLM), forming a sparse-activated mixture-of-experts model. This design enables the model to effectively adapt to a wide array of downstream tasks. Moreover, we employ a reinforcement learning-based training objective to train our model as a Q-function after deeply exploring the structural properties of our tasks. Effective learning from automatically collected mixed-quality data enhances data efficiency and model performance. Extensive experiments demonstrate that MoRE outperforms all baselines across six different skills and exhibits superior generalization capabilities in out-of-distribution scenarios. We further validate our method in real-world scenarios, confirming the practicality of our approach and laying a solid foundation for future research on multi-task learning in quadruped robots.
Code (0)
등록된 구현이 없습니다.
Tasks
Large Language ModelMixture-of-ExpertsMulti-Task Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Vision-Language-ActionSimilar Papers 제목 키워드 기반
Unlocking Pixels for Reinforcement Learning via Implicit Attention
There has recently been significant interest in training reinforcement learning (RL) agents in vision-based environments. This poses many challenges, such as high dimensionality and the potential for observational overfi…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Learning multiple gaits of quadruped robot using hierarchical reinforcement learning
There is a growing interest in learning a velocity command tracking controller of quadruped robot using reinforcement learning due to its robustness and scalability. However, a single policy, trained end-to-end, usually …
Hierarchical Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Constraint-Aware Diffusion Priors for High-Fidelity and Versatile Quadruped Locomotion
Reinforcement learning combined with imitation learning has significantly advanced biomimetic quadrupedal locomotion. However, scaling these frameworks to massive, multi-source datasets exposes fundamental bottlenecks. F…
Reinforcement LearningSigLoMa: Learning Open-World Quadrupedal Loco-Manipulation from Ego-Centric Vision
Designing an open-world quadrupedal loco-manipulation system is highly challenging. Traditional reinforcement learning frameworks utilizing exteroception often suffer from extreme sample inefficiency and massive sim-to-r…
Reinforcement LearningVisual TrackingAccelerating Robotic Reinforcement Learning with Agent Guidance
Reinforcement Learning (RL) offers a powerful paradigm for autonomous robots to master generalist manipulation skills through trial-and-error. However, its real-world application is stifled by low sample efficiency. Rece…
Reinforcement Learning