홈 › Papers

Podracer architectures for scalable Reinforcement Learning

2021-04-13 · Matteo Hessel, Manuel Kroiss, Aidan Clark, Iurii Kemaev, John Quan, Thomas Keck, Fabio Viola, Hado van Hasselt

Supporting state-of-the-art AI research requires balancing rapid prototyping, ease of use, and quick iteration, with the ability to deploy experiments at a scale traditionally associated with production systems.Deep learning frameworks such as TensorFlow, PyTorch and JAX allow users to transparently make use of accelerators, such as TPUs and GPUs, to offload the more computationally intensive parts of training and inference in modern deep learning systems. Popular training pipelines that use these frameworks for deep learning typically focus on (un-)supervised learning. How to best train reinforcement learning (RL) agents at scale is still an active research area. In this report we argue that TPUs are particularly well suited for training RL agents in a scalable, efficient and reproducible way. Specifically we describe two architectures designed to make the best use of the resources available on a TPU Pod (a special configuration in a Google data center that features multiple TPU devices connected to each other by extremely low latency communication channels).

📄 PDF Abstract BibTeX arXiv:2104.06272

Code (3)

instadeepai/flashbax jax

instadeepai/jumanji jax

sotetsuk/pgx jax

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning

2021-12-11 · Xiao-Yang Liu, Zechu Li, Zhuoran Yang, Jiahao Zheng 외

Deep reinforcement learning (DRL) has revolutionized learning and actuation in applications such as game playing and robotic control. The cost of data collection, i.e., generating transitions from agent-environment inter…

Deep Reinforcement Learning GPU reinforcement-learning Reinforcement Learning+3

FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance

2021-11-07 · Zechu Li, Xiao-Yang Liu, Jiahao Zheng, Zhaoran Wang 외

Machine learning techniques are playing more and more important roles in finance market investment. However, finance quantitative modeling with conventional supervised learning approaches has a number of limitations. The…

Deep Reinforcement Learning GPU reinforcement-learning Reinforcement Learning+3

IMPACT: Importance Weighted Asynchronous Architectures with Clipped Target Networks

2019-11-30 · ICLR 2020 1 · Michael Luo, Jiahao Yao, Richard Liaw, Eric Liang 외

The practical usage of reinforcement learning agents is often bottlenecked by the duration of training time. To accelerate training, practitioners often turn to distributed reinforcement learning architectures to paralle…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Scalable Reinforcement Learning-based Neural Architecture Search

2024-10-02 · Amber Cassimon, Siegfried Mercelis, Kevin Mets

In this publication, we assess the ability of a novel Reinforcement Learning-based solution to the problem of Neural Architecture Search, where a Reinforcement Learning (RL) agent learns to search for good architectures,…

Neural Architecture Search reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Minimal Value-Equivalent Partial Models for Scalable and Robust Planning in Lifelong Reinforcement Learning

2023-01-24 · Safa Alver, Doina Precup

Learning models of the environment from pure interaction is often considered an essential component of building lifelong reinforcement learning agents. However, the common practice in model-based reinforcement learning i…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)