From Pixels to Torques: Policy Learning with Deep Dynamical Models

2015-02-08 · Niklas Wahlström, Thomas B. Schön, Marc Peter Deisenroth

Data-efficient learning in continuous state-action spaces using very high-dimensional observations remains a key challenge in developing fully autonomous systems. In this paper, we consider one instance of this challenge, the pixels to torques problem, where an agent must learn a closed-loop control policy from pixel information only. We introduce a data-efficient, model-based reinforcement learning algorithm that learns such a closed-loop policy directly from pixel information. The key ingredient is a deep dynamical model that uses deep auto-encoders to learn a low-dimensional embedding of images jointly with a predictive model in this low-dimensional feature space. Joint learning ensures that not only static but also dynamic properties of the data are accounted for. This is crucial for long-term predictions, which lie at the core of the adaptive model predictive control strategy that we use for closed-loop control. Compared to state-of-the-art reinforcement learning methods for continuous states and actions, our approach learns quickly, scales to high-dimensional state spaces and is an important step toward fully autonomous learning from pixels to torques.

📄 PDF Abstract BibTeX arXiv:1502.02251

Code (0)

등록된 구현이 없습니다.

Tasks

Model-based Reinforcement LearningModel Predictive Controlreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Data-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Models

2015-10-08 · John-Alexander M. Assael, Niklas Wahlström, Thomas B. Schön, Marc Peter Deisenroth

Data-efficient reinforcement learning (RL) in continuous state-action spaces using very high-dimensional observations remains a key challenge in developing fully autonomous systems. We consider a particularly important i…

Model-based Reinforcement Learning Model Predictive Control reinforcement-learning Reinforcement Learning+1

HybridMimic: Hybrid RL-Centroidal Control for Humanoid Motion Mimicking

2026-03-06 · Ludwig Chee-Ying Tay, I-Chia Chang, Yan Gu arxiv

Motion mimicking, i.e., encouraging the control policy to mimic human motion, facilitates the learning of complex tasks via reinforcement learning (RL) for humanoid robots. Although standard RL frameworks demonstrate imp…

Reinforcement Learning

Self-supervised Learning of Image Embedding for Continuous Control

2019-01-03 · Carlos Florensa, Jonas Degrave, Nicolas Heess, Jost Tobias Springenberg 외

Operating directly from raw high dimensional sensory inputs like images is still a challenge for robotic control. Recently, Reinforcement Learning methods have been proposed to solve specific tasks end-to-end, from pixel…

continuous-control Continuous Control Reinforcement Learning Self-Supervised Learning

Wild Motion Unleashed: Markerless 3D Kinematics and Force Estimation in Cheetahs

2023-12-10 · Zico da Silva, Stacy Shield, Penny E. Hudson, Alan M. Wilson 외

The complex dynamics of animal manoeuvrability in the wild is extremely challenging to study. The cheetah ($\textit{Acinonyx jubatus}$) is a perfect example: despite great interest in its unmatched speed and manoeuvrabil…

SimPoE: Simulated Character Control for 3D Human Pose Estimation

2021-04-01 · CVPR 2021 1 · Ye Yuan, Shih-En Wei, Tomas Simon, Kris Kitani 외

Accurate estimation of 3D human motion from monocular video requires modeling both kinematics (body motion without physical forces) and dynamics (motion with physical forces). To demonstrate this, we present SimPoE, a Si…

3D Human Pose Estimation Pose Estimation