Evolving Constrained Reinforcement Learning Policy

2023-04-19 · Chengpeng Hu, Jiyuan Pei, Jialin Liu, Xin Yao

Evolutionary algorithms have been used to evolve a population of actors to generate diverse experiences for training reinforcement learning agents, which helps to tackle the temporal credit assignment problem and improves the exploration efficiency. However, when adapting this approach to address constrained problems, balancing the trade-off between the reward and constraint violation is hard. In this paper, we propose a novel evolutionary constrained reinforcement learning (ECRL) algorithm, which adaptively balances the reward and constraint violation with stochastic ranking, and at the same time, restricts the policy's behaviour by maintaining a set of Lagrange relaxation coefficients with a constraint buffer. Extensive experiments on robotic control benchmarks show that our ECRL achieves outstanding performance compared to state-of-the-art algorithms. Ablation analysis shows the benefits of introducing stochastic ranking and constraint buffer.

📄 PDF Abstract BibTeX arXiv:2304.09869

Code (1)

hcplu/evolutionary-constrained-reinforcement-learning 공식 구현 pytorch

Tasks

Evolutionary Algorithmsreinforcement-learning Reinforcement Learning

AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learning

2023-01-24 · Tairan He, WeiYe Zhao, Changliu Liu

Safety is a critical hurdle that limits the application of deep reinforcement learning (RL) to real-world control tasks. To this end, constrained reinforcement learning leverages cost functions to improve safety in const…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Safe and Efficient Self-evolving Algorithm for Decision-making and Control of Autonomous Driving Systems

2024-08-22 · Shuo Yang, LiWen Wang, Yanjun Huang, Hong Chen

Autonomous vehicles with a self-evolving ability are expected to cope with unknown scenarios in the real-world environment. Take advantage of trial and error mechanism, reinforcement learning is able to self evolve by le…

Autonomous Driving Autonomous Vehicles Decision Making reinforcement-learning+1

ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiring

2026-05-04 · Deyi Ji, Junyu Lu, Xuanyi Liu, Liqun Liu 외 arxiv

Online advertising governance faces significant challenges due to the non-stationary nature of regulatory policies, where emerging mandates (e.g., restrictions on education or aesthetic anxiety) create severe label incon…

Reinforcement Learning

Integrated Decision and Control for High-Level Automated Vehicles by Mixed Policy Gradient and Its Experiment Verification

2022-10-19 · Yang Guan, Liye Tang, Chuanxiao Li, Shengbo Eben Li 외

Self-evolution is indispensable to realize full autonomous driving. This paper presents a self-evolving decision-making system based on the Integrated Decision and Control (IDC), an advanced framework built on reinforcem…

Autonomous Driving Decision Making Reinforcement Learning (RL)

Adversarial Constrained Policy Optimization: Improving Constrained Reinforcement Learning by Adapting Budgets

2024-10-28 · Jianmina Ma, Jingtian Ji, Yue Gao

Constrained reinforcement learning has achieved promising progress in safety-critical fields where both rewards and constraints are considered. However, constrained reinforcement learning methods face challenges in strik…

reinforcement-learning Reinforcement Learning