Context-Aware Safe Reinforcement Learning for Non-Stationary Environments
Safety is a critical concern when deploying reinforcement learning agents for realistic tasks. Recently, safe reinforcement learning algorithms have been developed to optimize the agent's performance while avoiding violations of safety constraints. However, few studies have addressed the non-stationary disturbances in the environments, which may cause catastrophic outcomes. In this paper, we propose the context-aware safe reinforcement learning (CASRL) method, a meta-learning framework to realize safe adaptation in non-stationary environments. We use a probabilistic latent variable model to achieve fast inference of the posterior environment transition distribution given the context data. Safety constraints are then evaluated with uncertainty-aware trajectory sampling. The high cost of safety violations leads to the rareness of unsafe records in the dataset. We address this issue by enabling prioritized sampling during model training and formulating prior safety constraints with domain knowledge during constrained planning. The algorithm is evaluated in realistic safety-critical environments with non-stationary disturbances. Results show that the proposed algorithm significantly outperforms existing baselines in terms of safety and robustness.
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Safe Reinforcement LearningSimilar Papers 제목 키워드 기반
Safe Continual Reinforcement Learning under Nonstationarity via Adaptive Safety Constraints
Safe reinforcement learning in nonstationary environments requires safety mechanisms that adapt as environmental conditions change. Standard safe reinforcement learning methods often assume fixed constraints or stable en…
Reinforcement LearningSafe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Art
This work provides a state-of-the-art survey of continual safe online reinforcement learning (COSRL) methods. We discuss theoretical aspects, challenges, and open questions in building continual online safe reinforcement…
Reinforcement LearningContinual LearningSafe Continual Reinforcement Learning in Non-stationary Environments
Reinforcement learning (RL) offers a compelling data-driven paradigm for synthesizing controllers for complex systems when accurate physical models are unavailable; however, most existing control-oriented RL methods assu…
Reinforcement LearningGraph-based Prediction and Planning Policy Network (GP3Net) for scalable self-driving in dynamic environments using Deep Reinforcement Learning
Recent advancements in motion planning for Autonomous Vehicles (AVs) show great promise in using expert driver behaviors in non-stationary driving environments. However, learning only through expert drivers needs more ge…
Autonomous VehiclesBenchmarkingDeep Reinforcement LearningImitation Learning+1Optimal Dynamic Regret by Transformers for Non-Stationary Reinforcement Learning
Transformers have demonstrated exceptional performance across a wide range of domains. While their ability to perform reinforcement learning in-context has been established both theoretically and empirically, their behav…
Reinforcement Learning