Proximal Policy Optimization for Tracking Control Exploiting Future Reference Information
In recent years, reinforcement learning (RL) has gained increasing attention in control engineering. Especially, policy gradient methods are widely used. In this work, we improve the tracking performance of proximal policy optimization (PPO) for arbitrary reference signals by incorporating information about future reference values. Two variants of extending the argument of the actor and the critic taking future reference values into account are presented. In the first variant, global future reference values are added to the argument. For the second variant, a novel kind of residual space with future reference values applicable to model-free reinforcement learning is introduced. Our approach is evaluated against a PI controller on a simple drive train model. We expect our method to generalize to arbitrary references better than previous approaches, pointing towards the applicability of RL to control real systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Adaptive Gain Scheduling using Reinforcement Learning for Quadcopter Control
The paper presents a technique using reinforcement learning (RL) to adapt the control gains of a quadcopter controller. Specifically, we employed Proximal Policy Optimization (PPO) to train a policy which adapts the gain…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)SchedulingLeveraging Symmetry to Accelerate Learning of Trajectory Tracking Controllers for Free-Flying Robotic Systems
Tracking controllers enable robotic systems to accurately follow planned reference trajectories. In particular, reinforcement learning (RL) has shown promise in the synthesis of controllers for systems with complex dynam…
Reinforcement Learning (RL)Real-Time Model-Free Deep Reinforcement Learning for Force Control of a Series Elastic Actuator
Many state-of-the art robotic applications utilize series elastic actuators (SEAs) with closed-loop force control to achieve complex tasks such as walking, lifting, and manipulation. Model-free PID control methods are mo…
continuous-controlContinuous ControlDeep Reinforcement LearningDeep Gaussian Process Proximal Policy Optimization
Uncertainty estimation for Reinforcement Learning (RL) is a critical component in control tasks where agents must balance safe exploration and efficient learning. While deep neural networks have enabled breakthroughs in …
Reinforcement LearningContinuous ControlGaussian ProcessesTaking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control
In this work we introduce the application of black-box quantum control as an interesting rein- forcement learning problem to the machine learning community. We analyze the structure of the reinforcement learning problems…
Reinforcement Learning