Asymmetric self-play for automatic goal discovery in robotic manipulation
We train a single, goal-conditioned policy that can solve many robotic manipulation tasks, including tasks with previously unseen goals and objects. We rely on asymmetric self-play for goal discovery, where two agents, Alice and Bob, play a game. Alice is asked to propose challenging goals and Bob aims to solve them. We show that this method can discover highly diverse and complex goals without any human priors. Bob can be trained with only sparse rewards, because the interaction between Alice and Bob results in a natural curriculum and Bob can learn from Alice's trajectory when relabeled as a goal-conditioned demonstration. Finally, our method scales, resulting in a single policy that can generalize to many unseen tasks such as setting a table, stacking blocks, and solving simple puzzles. Videos of a learned policy is available at https://robotics-self-play.github.io.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
GASP: Guided Asymmetric Self-Play For Coding LLMs
Asymmetric self-play has emerged as a promising paradigm for post-training large language models, where a teacher continually generates questions for a student to solve at the edge of the student's learnability. Although…
It Takes Four to Tango: Multiagent Self Play for Automatic Curriculum Generation
We are interested in training general-purpose reinforcement learning agents that can solve a wide variety of goals. Training such agents without excessive interactions requires automatic generation of a goal curriculum.…
Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning
In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automa…
Hierarchical Reinforcement LearningMuJoCoreinforcement-learningReinforcement Learning+1It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generation
We are interested in training general-purpose reinforcement learning agents that can solve a wide variety of goals. Training such agents efficiently requires automatic generation of a goal curriculum. This is challenging…
SPARK: Self-Play with Asymmetric Reward from Knowledge Graphs
Self-play reinforcement learning has shown strong performance in domains with formally verifiable structure, such as mathematics and coding, where both problem generation and reward computation can be grounded in explici…
Reinforcement LearningRelational ReasoningKnowledge Graphs