Implicit Quantile Networks for Distributional Reinforcement Learning

2018-06-14 · ICML 2018 7 · Will Dabney, Georg Ostrovski, David Silver, Rémi Munos

In this work, we build on recent advances in distributional reinforcement learning to give a generally applicable, flexible, and state-of-the-art distributional variant of DQN. We achieve this by using quantile regression to approximate the full quantile function for the state-action return distribution. By reparameterizing a distribution over the sample space, this yields an implicitly defined return distribution and gives rise to a large class of risk-sensitive policies. We demonstrate improved performance on the 57 Atari 2600 games in the ALE, and use our algorithm's implicitly defined distributions to study the effects of risk-sensitive policies in Atari games.

📄 PDF Abstract BibTeX arXiv:1806.06923

Code (19)

ACampero/dopamine tf

Abdelhamid-bouzid/Distributional-RL pytorch

BY571/IQN pytorch

GoingMyWay/dopamine_reward_decomposition tf

KatyNTsachi/Hierarchical-RL tf

Kchu/DeepRL_CK pytorch

SeungeonBaek/discrete-agents-test tf

V0LsTeR/DQN_heap tf

V0LsTeR/dopamine_prioritized_buffer tf

chainer/chainerrl pytorch

ku2482/fqf-iqn-qrdqn.pytorch pytorch

ku2482/rljax jax

marload/DistRL-TensorFlow2 tf

marload/dist-rl-tf2 tf

opendilab/DI-engine/blob/main/ding/policy/iqn.py pytorch

pihey1995/DistributionalRL pytorch

robinzixuan/IQN_Agent pytorch

sjYoondeltar/IQN_example tf

sjYoondeltar/myRL_example tf

Tasks

Atari GamesDistributional Reinforcement Learningquantile regressionregression reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Quantile Geometry Regularization for Distributional Reinforcement Learning

2026-05-05 · Zhaofan Zhang, Minghao Yang, Rufeng Chen, Sihong Xie 외 arxiv

Quantile-based distributional reinforcement learning methods learn return distributions through sampled quantile regression, but their bootstrapped target quantiles may induce distorted or degenerate distribution estimat…

Reinforcement Learning Atari Games

IGN : Implicit Generative Networks

2022-06-13 · Haozheng Luo, Tianyi Wu, Feiyu Han, Zhijun Yan 외

In this work, we build recent advances in distributional reinforcement learning to give a state-of-art distributional variant of the model based on the IQN. We achieve this by using the GAN model's generator and discrimi…

Atari Games Distributional Reinforcement Learning quantile regression regression+2

PG-Rainbow: Using Distributional Reinforcement Learning in Policy Gradient Methods

2024-07-18 · WooJae Jeon, KangJun Lee, Jeewoo Lee

This paper introduces PG-Rainbow, a novel algorithm that incorporates a distributional reinforcement learning framework with a policy gradient algorithm. Existing policy gradient methods are sample inefficient and rely o…

Atari Games Decision Making Distributional Reinforcement Learning Policy Gradient Methods+2

Distributional Reinforcement Learning with Monotonic Splines

2021-09-29 · ICLR 2022 4 · Yudong Luo, Guiliang Liu, Haonan Duan, Oliver Schulte 외

Distributional Reinforcement Learning (RL) differs from traditional RL by estimating the distribution over returns to capture the intrinsic uncertainty of MDPs. One key challenge in distributional RL lies in how to param…

Distributional Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Fully Parameterized Quantile Function for Distributional Reinforcement Learning

2019-11-05 · NeurIPS 2019 12 · Derek Yang, Li Zhao, Zichuan Lin, Tao Qin 외

Distributional Reinforcement Learning (RL) differs from traditional RL in that, rather than the expectation of total returns, it estimates distributions and has achieved state-of-the-art performance on Atari Games. The k…

Atari Games Distributional Reinforcement Learning reinforcement-learning Reinforcement Learning+1