Enhance the Safety in Reinforcement Learning by ADRC Lagrangian Methods
Safe reinforcement learning (Safe RL) seeks to maximize rewards while satisfying safety constraints, typically addressed through Lagrangian-based methods. However, existing approaches, including PID and classical Lagrangian methods, suffer from oscillations and frequent safety violations due to parameter sensitivity and inherent phase lag. To address these limitations, we propose ADRC-Lagrangian methods that leverage Active Disturbance Rejection Control (ADRC) for enhanced robustness and reduced oscillations. Our unified framework encompasses classical and PID Lagrangian methods as special cases while significantly improving safety performance. Extensive experiments demonstrate that our approach reduces safety violations by up to 74%, constraint violation magnitudes by 89%, and average costs by 67\%, establishing superior effectiveness for Safe RL in complex environments.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
Model-based Chance-Constrained Reinforcement Learning via Separated Proportional-Integral Lagrangian
Safety is essential for reinforcement learning (RL) applied in the real world. Adding chance constraints (or probabilistic constraints) is a suitable way to enhance RL safety under uncertainty. Existing chance-constraine…
reinforcement-learningReinforcement Learning (RL)Robot NavigationSeparated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning
Safety is essential for reinforcement learning (RL) applied in real-world tasks like autonomous driving. Chance constraints which guarantee the satisfaction of state constraints at a high probability are suitable to repr…
Autonomous Drivingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Responsive Safety in Reinforcement Learning by PID Lagrangian Methods
Lagrangian methods are widely used algorithms for constrained optimization problems, but their learning dynamics exhibit oscillations and overshoot which, when applied to safe reinforcement learning, leads to constraint-…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe Reinforcement LearningLagrangian-based online safe reinforcement learning for state-constrained systems
This paper proposes a safe reinforcement learning (RL) algorithm that approximately solves the state-constrained optimal control problem for continuous-time uncertain nonlinear systems. We formulate the safe RL problem a…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe Reinforcement LearningTowards a Practical Understanding of Lagrangian Methods in Safe Reinforcement Learning
Safe reinforcement learning addresses constrained optimization problems where maximizing performance must be balanced against safety constraints, and Lagrangian methods are a widely used approach for this purpose. Howeve…
Reinforcement Learning