홈 › Papers

Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills

2021-04-15 · Yevgen Chebotar, Karol Hausman, Yao Lu, Ted Xiao, Dmitry Kalashnikov, Jake Varley, Alex Irpan, Benjamin Eysenbach, Ryan Julian, Chelsea Finn, Sergey Levine

We consider the problem of learning useful robotic skills from previously collected offline data without access to manually specified rewards or additional online exploration, a setting that is becoming increasingly important for scaling robot learning by reusing past robotic data. In particular, we propose the objective of learning a functional understanding of the environment by learning to reach any goal state in a given dataset. We employ goal-conditioned Q-learning with hindsight relabeling and develop several techniques that enable training in a particularly challenging offline setting. We find that our method can operate on high-dimensional camera images and learn a variety of skills on real robots that generalize to previously unseen scenes and objects. We also show that our method can learn to reach long-horizon goals across multiple episodes through goal chaining, and learn rich representations that can help with downstream tasks through pre-training or auxiliary objectives. The videos of our experiments can be found at https://actionable-models.github.io

📄 PDF Abstract BibTeX arXiv:2104.07749

Code (0)

등록된 구현이 없습니다.

Tasks

Q-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Implicit Offline Reinforcement Learning via Supervised Learning

2022-10-21 · Alexandre Piche, Rafael Pardinas, David Vazquez, Igor Mordatch 외

Offline Reinforcement Learning (RL) via Supervised Learning is a simple and effective way to learn robotic skills from a dataset collected by policies of different expertise levels. It is as simple as supervised learning…

Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Offline Skill Graph (OSG): A Framework for Learning and Planning using Offline Reinforcement Learning Skills

2023-06-23 · Ben-ya Halevy, Yehudit Aperstein, Dotan Di Castro

Reinforcement Learning has received wide interest due to its success in competitive games. Yet, its adoption in everyday applications is limited (e.g. industrial, home, healthcare, etc.). In this paper, we address this l…

reinforcement-learning

Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning

2020-04-27 · Archit Sharma, Michael Ahn, Sergey Levine, Vikash Kumar 외

Reinforcement learning provides a general framework for learning robotic skills while minimizing engineering effort. However, most reinforcement learning algorithms assume that a well-designed reward function is provided…

Model Predictive Control reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Offline Learning of Counterfactual Predictions for Real-World Robotic Reinforcement Learning

2020-11-11 · Jun Jin, Daniel Graves, Cameron Haigh, Jun Luo 외

We consider real-world reinforcement learning (RL) of robotic manipulation tasks that involve both visuomotor skills and contact-rich skills. We aim to train a policy that maps multimodal sensory observations (vision and…

counterfactual reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning

2020-10-27 · Avi Singh, Albert Yu, Jonathan Yang, Jesse Zhang 외

Reinforcement learning has been applied to a wide variety of robotics problems, but most of such applications involve collecting data from scratch for each new task. Since the amount of robot data we can collect for any …

reinforcement-learning Reinforcement Learning (RL)