Hysteresis-Based RL: Robustifying Reinforcement Learning-based Control Policies via Hybrid Control

2022-04-01 · Jan de Priester, Ricardo G. Sanfelice, Nathan van de Wouw

Reinforcement learning (RL) is a promising approach for deriving control policies for complex systems. As we show in two control problems, the derived policies from using the Proximal Policy Optimization (PPO) and Deep Q-Network (DQN) algorithms may lack robustness guarantees. Motivated by these issues, we propose a new hybrid algorithm, which we call Hysteresis-Based RL (HyRL), augmenting an existing RL algorithm with hysteresis switching and two stages of learning. We illustrate its properties in two examples for which PPO and DQN fail.

📄 PDF Abstract BibTeX arXiv:2204.00654

Code (2)

hybridsystemslab/obstacleavoidancehyrl 공식 구현 pytorch

hybridsystemslab/unitcirclehyrl 공식 구현 pytorch

Tasks

reinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Entropy Regularization 설명 없음

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Robustifying Reinforcement Learning Policies with $\mathcal{L}_1$ Adaptive Control

2021-06-04 · Yikun Cheng, Pan Zhao, Manan Gandhi, Bo Li 외

A reinforcement learning (RL) policy trained in a nominal environment could fail in a new/perturbed environment due to the existence of dynamic variations. Existing robust methods try to obtain a fixed policy for all env…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Evolving LLM-Derived Control Policies for Residential EV Charging and Vehicle-to-Grid Energy Optimization

2026-02-06 · Vishesh Purnananda, Benjamin John Wruck, Mingyu Guo arxiv

This research presents a novel application of Evolutionary Computation to the domain of residential electric vehicle (EV) energy management. While reinforcement learning (RL) achieves high performance in vehicle-to-grid …

Reinforcement Learning

A Physics-Based Hybrid Dynamical Model of Hysteresis in Polycrystalline Shape Memory Alloy Wire Transducers

2023-05-23 · M. Mandolino, D. Scholtes, F. Ferrante, G. Rizzello

Shape Memory Alloys (SMAs) are a class of smart materials that exhibit a macroscopic contraction of up to 5% when heated via an electric current. This effect can be exploited for the development of novel unconventional a…

A reinforcement learning approach to hybrid control design

2020-09-02 · Meet Gandhi, Atreyee Kundu, Shalabh Bhatnagar

In this paper we design hybrid control policies for hybrid systems whose mathematical models are unknown. Our contributions are threefold. First, we propose a framework for modelling the hybrid control design problem as …