Is Vanilla Policy Gradient Overlooked? Analyzing Deep Reinforcement Learning for Hanabi

2022-03-22 · Bram Grooten, Jelle Wemmenhove, Maurice Poot, Jim Portegies

In pursuit of enhanced multi-agent collaboration, we analyze several on-policy deep reinforcement learning algorithms in the recently published Hanabi benchmark. Our research suggests a perhaps counter-intuitive finding, where Proximal Policy Optimization (PPO) is outperformed by Vanilla Policy Gradient over multiple random seeds in a simplified environment of the multi-agent cooperative card game. In our analysis of this behavior we look into Hanabi-specific metrics and hypothesize a reason for PPO's plateau. In addition, we provide proofs for the maximum length of a perfect game (71 turns) and any game (89 turns). Our code can be found at: https://github.com/bramgrooten/DeepRL-for-Hanabi

📄 PDF Abstract BibTeX arXiv:2203.11656

Code (1)

bramgrooten/deeprl-for-hanabi 공식 구현 pytorch

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning (RL)

Theoretical Guarantees of Fictitious Discount Algorithms for Episodic Reinforcement Learning and Global Convergence of Policy Gradient Methods

2021-09-13 · Xin Guo, Anran Hu, Junzi Zhang

When designing algorithms for finite-time-horizon episodic reinforcement learning problems, a common approach is to introduce a fictitious discount factor and use stationary policies for approximations. Empirically, it h…

Policy Gradient Methods reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Future Policy Approximation for Offline Reinforcement Learning in LLM Reasoning

2025-09-24 · Minjae Oh, Yunho Choi, Dongmin Choi, Yohan Jo arxiv

Reinforcement learning (RL) has emerged as a key driver of post-training for complex reasoning in large language models (LLMs), yet online RL introduces substantial instability and computational overhead. Offline RL offe…

Reinforcement Learning Mathematical Reasoning Offline RL

Variational Inference for Policy Gradient

2018-02-21 · Tianbing Xu

Inspired by the seminal work on Stein Variational Inference and Stein Variational Policy Gradient, we derived a method to generate samples from the posterior variational parameter distribution by \textit{explicitly} mini…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Variational Inference

Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Method

2023-09-21 · NeurIPS 2023 11

Deep Neural Networks and Reinforcement Learning methods have empirically shown great promise in tackling challenging combinatorial problems. In those methods a deep neural network is used as a solution generator which is…

Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods

2023-10-08 · Constantine Caramanis, Dimitris Fotakis, Alkis Kalavasis, Vasilis Kontonis 외

Policy Gradient Methods Traveling Salesman Problem