Comparative Analysis of Parameterized Action Actor-Critic Reinforcement Learning Algorithms for Web Search Match Plan Generation

2025-10-03 · Ubayd Bapoo, Clement N Nyirenda arxiv

This study evaluates the performance of Soft Actor Critic (SAC), Greedy Actor Critic (GAC), and Truncated Quantile Critics (TQC) in high-dimensional decision-making tasks using fully observable environments. The focus is on parametrized action (PA) spaces, eliminating the need for recurrent networks, with benchmarks Platform-v0 and Goal-v0 testing discrete actions linked to continuous action-parameter spaces. Hyperparameter optimization was performed with Microsoft NNI, ensuring reproducibility by modifying the codebase for GAC and TQC. Results show that Parameterized Action Greedy Actor-Critic (PAGAC) outperformed other algorithms, achieving the fastest training times and highest returns across benchmarks, completing 5,000 episodes in 41:24 for the Platform game and 24:04 for the Robot Soccer Goal game. Its speed and stability provide clear advantages in complex action spaces. Compared to PASAC and PATQC, PAGAC demonstrated superior efficiency and reliability, making it ideal for tasks requiring rapid convergence and robust performance. Future work could explore hybrid strategies combining entropy-regularization with truncation-based methods to enhance stability and expand investigations into generalizability.

📄 PDF Abstract BibTeX arXiv:2510.03064

Code (0)

등록된 구현이 없습니다.

Tasks

Hyperparameter Optimization Reinforcement Learning

Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Space

2019-03-04 · Zhou Fan, Rui Su, Wei-Nan Zhang, Yong Yu

In this paper we propose a hybrid architecture of actor-critic algorithms for reinforcement learning in parameterized action space, which consists of multiple parallel sub-actor networks to decompose the structured actio…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for Discretionary Lane Change

2024-03-01 · Ruichen Xu, Xiao Liu, Jinming Xu, Yuan Lin

Autonomous lane-change, a key feature of advanced driver-assistance systems, can enhance traffic efficiency and reduce the incidence of accidents. However, safe driving of autonomous vehicles remains challenging in compl…

Autonomous Driving Autonomous Vehicles reinforcement-learning

Neural Policy Gradient Methods: Global Optimality and Rates of Convergence

2019-08-29 · ICLR 2020 1 · Lingxiao Wang, Qi Cai, Zhuoran Yang, Zhaoran Wang

Policy gradient methods with actor-critic schemes demonstrate tremendous empirical successes, especially when the actors and critics are parameterized by neural networks. However, it remains less clear whether such "neur…

Policy Gradient Methods

Guide Actor-Critic for Continuous Control

2017-05-22 · ICLR 2018 1 · Voot Tangkaratt, Abbas Abdolmaleki, Masashi Sugiyama

Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Greedy Actor-Critic: A New Conditional Cross-Entropy Method for Policy Improvement

2018-10-22 · Samuel Neumann, Sungsu Lim, Ajin Joseph, Yangchen Pan 외

Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update…

Policy Gradient Methods Q-Learning