A Safety Modulator Actor-Critic Method in Model-Free Safe Reinforcement Learning and Application in UAV Hovering
This paper proposes a safety modulator actor-critic (SMAC) method to address safety constraint and overestimation mitigation in model-free safe reinforcement learning (RL). A safety modulator is developed to satisfy safety constraints by modulating actions, allowing the policy to ignore safety constraint and focus on maximizing reward. Additionally, a distributional critic with a theoretical update rule for SMAC is proposed to mitigate the overestimation of Q-values with safety constraints. Both simulation and real-world scenarios experiments on Unmanned Aerial Vehicles (UAVs) hovering confirm that the SMAC can effectively maintain safety constraints and outperform mainstream baseline algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement Learning (RL)Safe Reinforcement LearningSMACSMAC+Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Multiplicative Value Function for Safe and Efficient Reinforcement Learning
An emerging field of sequential decision problems is safe Reinforcement Learning (RL), where the objective is to maximize the reward while obeying safety constraints. Being able to handle constraints is essential for dep…
Navigatereinforcement-learningReinforcement LearningReinforcement Learning (RL)+2Robust Koopman Control Barrier Filters for Safe Actor-Critic Reinforcement Learning
Safe reinforcement learning (RL) for robotic systems requires policies that improve task performance while satisfying state and input constraints during both training and deployment. Control barrier functions (CBFs) prov…
Reinforcement LearningFeasible Actor-Critic: Constrained Reinforcement Learning for Ensuring Statewise Safety
The safety constraints commonly used by existing safe reinforcement learning (RL) methods are defined only on expectation of initial states, but allow each certain state to be unsafe, which is unsatisfying for real-world…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe Exploration+1Optimal Variable Speed Limit Control Strategy on Freeway Segments under Fog Conditions
Fog is a critical external factor that threatens traffic safety on freeways. Variable speed limit (VSL) control can effectively harmonize vehicle speed and improve safety. However, most existing weather-related VSL contr…
Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions
Reinforcement learning (RL) exhibits impressive performance when managing complicated control tasks for robots. However, its wide application to physical robots is limited by the absence of strong safety guarantees. To o…
reinforcement-learningReinforcement Learning (RL)