Consistent Dropout for Policy Gradient Reinforcement Learning

2022-02-23 · Matthew Hausknecht, Nolan Wagener

Dropout has long been a staple of supervised learning, but is rarely used in reinforcement learning. We analyze why naive application of dropout is problematic for policy-gradient learning algorithms and introduce consistent dropout, a simple technique to address this instability. We demonstrate consistent dropout enables stable training with A2C and PPO in both continuous and discrete action environments across a wide range of dropout probabilities. Finally, we show that consistent dropout enables the online training of complex architectures such as GPT without needing to disable the model's native dropout.

📄 PDF Abstract BibTeX arXiv:2202.11818

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Entropy Regularization 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Discriminative Fine-Tuning Discriminative Fine-Tuning is a fine-tuning strategy that is used for ULMFiT type models. Instead of using the same learning rate…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

NADPEx: An on-policy temporally consistent exploration method for deep reinforcement learning

2018-12-21 · ICLR 2019 5 · Sirui Xie, Junning Huang, Lanxin Lei, Chunxiao Liu 외

Reinforcement learning agents need exploratory behaviors to escape from local optima. These behaviors may include both immediate dithering perturbation and temporally consistent exploration. To achieve these, a stochasti…

continuous-control Continuous Control Deep Reinforcement Learning MuJoCo+3

Policy Gradient Guidance Enables Test Time Control

2025-10-02 · Jianing Qi, Hao Tang, Zhigang Zhu arxiv

We introduce Policy Gradient Guidance (PGG), a simple extension of classifier-free guidance from diffusion models to classical policy gradient methods. PGG augments the policy gradient with an unconditional branch and in…

Reinforcement Learning Continuous Control

MEPG: A Minimalist Ensemble Policy Gradient Framework for Deep Reinforcement Learning

2021-09-22 · Qiang He, Huangyuan Su, Chen Gong, Xinwen Hou

During the training of a reinforcement learning (RL) agent, the distribution of training data is non-stationary as the agent's behavior changes over time. Therefore, there is a risk that the agent is overspecialized to a…

Deep Reinforcement Learning Gaussian Processes Q-Learning reinforcement-learning+1

Message-Dropout: An Efficient Training Method for Multi-Agent Deep Reinforcement Learning

2019-02-18 · Woojun Kim, Myungsik Cho, Youngchul Sung

In this paper, we propose a new learning technique named message-dropout to improve the performance for multi-agent deep reinforcement learning under two application scenarios: 1) classical multi-agent reinforcement lear…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Black-Box Combinatorial Optimization with Order-Invariant Reinforcement Learning

2025-10-02 · Olivier Goudet, Quentin Suire, Adrien Goëffon, Frédéric Saubion 외 arxiv

We introduce an order-invariant reinforcement learning framework for black-box combinatorial optimization. Classical estimation-of-distribution algorithms (EDAs) often rely on learning explicit variable dependency graphs…

Reinforcement Learning