Learn Proportional Derivative Controllable Latent Space from Pixels
Recent advances in latent space dynamics model from pixels show promising progress in vision-based model predictive control (MPC). However, executing MPC in real time can be challenging due to its intensive computational cost in each timestep. We propose to introduce additional learning objectives to enforce that the learned latent space is proportional derivative controllable. In execution time, the simple PD-controller can be applied directly to the latent space encoded from pixels, to produce simple and effective control to systems with visual observations. We show that our method outperforms baseline methods to produce robust goal reaching and trajectory tracking in various environments.
Code (0)
등록된 구현이 없습니다.
Tasks
Model Predictive ControlSimilar Papers 제목 키워드 기반
Rethinking Controllable Variational Autoencoders
The Controllable Variational Autoencoder (ControlVAE) combines automatic control theory with the basic VAE model to manipulate the KL-divergence for overcoming posterior collapse and learning disentangled representat…
DisentanglementImage GenerationLanguage ModelingLanguage Modelling+1An alternative to proportional-integral and proportional-integral-derivative regulators: Intelligent proportional-derivative regulators
This paper suggests to replace PIs and PIDs, which play a key role in control engineering, by intelligent Proportional-Derivative feedback loops, or iPDs, which are derived from model-free control. This standpoint is enh…
NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spaces
Learning low-dimensional latent state space dynamics models has been a powerful paradigm for enabling vision-based planning and learning for control. We introduce a latent dynamics learning framework that is uniquely des…
Behavioural cloningImitation LearningThe derivatives of Sinkhorn-Knopp converge
We show that the derivatives of the Sinkhorn-Knopp algorithm, or iterative proportional fitting procedure, converge towards the derivatives of the entropic regularization of the optimal transport problem with a locally u…
From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation
While controllable image generation has made significant strides by incorporating visual reference conditions, existing methods predominantly operate as open-loop systems. They inject control signals in a strictly feed-f…
Image Generation