Low-Precision Reinforcement Learning: Running Soft Actor-Critic in Half Precision

2021-02-26 · Johan Bjorck, Xiangyu Chen, Christopher De Sa, Carla P. Gomes, Kilian Q. Weinberger

Low-precision training has become a popular approach to reduce compute requirements, memory footprint, and energy consumption in supervised learning. In contrast, this promising approach has not yet enjoyed similarly widespread adoption within the reinforcement learning (RL) community, partly because RL agents can be notoriously hard to train even in full precision. In this paper we consider continuous control with the state-of-the-art SAC agent and demonstrate that a na\"ive adaptation of low-precision methods from supervised learning fails. We propose a set of six modifications, all straightforward to implement, that leaves the underlying agent and its hyperparameters unchanged but improves the numerical stability dramatically. The resulting modified SAC agent has lower memory and compute requirements while matching full-precision rewards, demonstrating that low-precision training can substantially accelerate state-of-the-art RL without parameter tuning.

📄 PDF Abstract BibTeX arXiv:2102.13565

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control reinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Average Pooling 설명 없음

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Dilated Convolution 설명 없음

SAC 설명 없음

Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropy

2022-06-10 · Hongzhi Hua, Kaigui Wu, Guixuan Wen

Multi-agent deep reinforcement learning has been applied to address a variety of complex problems with either discrete or continuous action spaces and achieved great success. However, most real-world environments cannot …

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Adviser-Actor-Critic: Eliminating Steady-State Error in Reinforcement Learning Control

2025-02-04 · Donghe Chen, Yubin Peng, Tengjie Zheng, Han Wang 외

High-precision control tasks present substantial challenges for reinforcement learning (RL) algorithms, frequently resulting in suboptimal performance attributed to network approximation inaccuracies and inadequate sampl…

Reinforcement Learning (RL)

Variational Quantum Soft Actor-Critic

2021-12-20 · Qingfeng Lan

Quantum computing has a superior advantage in tackling specific problems, such as integer factorization and Simon's problem. For more general tasks in machine learning, by applying variational quantum circuits, more and …

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

2018-01-04 · ICML 2018 7 · Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, Sergey Levine

A platform for Applied Reinforcement Learning (Applied RL)

Continuous Control Decision Making Deep Reinforcement Learning Omniverse Isaac Gym+5

A further exploration of deep Multi-Agent Reinforcement Learning with Hybrid Action Space

2022-08-30 · Hongzhi Hua, Guixuan Wen, Kaigui Wu

The research of extending deep reinforcement learning (drl) to multi-agent field has solved many complicated problems and made great achievements. However, almost all these studies only focus on discrete or continuous ac…