Temporal Difference Weighted Ensemble For Reinforcement Learning

2019-09-25 · Takuma Seno, Michita Imai

Combining multiple function approximators in machine learning models typically leads to better performance and robustness compared with a single function. In reinforcement learning, ensemble algorithms such as an averaging method and a majority voting method are not always optimal, because each function can learn fundamentally different optimal trajectories from exploration. In this paper, we propose a Temporal Difference Weighted (TDW) algorithm, an ensemble method that adjusts weights of each contribution based on accumulated temporal difference errors. The advantage of this algorithm is that it improves ensemble performance by reducing weights of Q-functions unfamiliar with current trajectories. We provide experimental results for Gridworld tasks and Atari tasks that show significant performance improvements compared with baseline algorithms.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Off-Policy Shaping Ensembles in Reinforcement Learning

2014-05-21 · Anna Harutyunyan, Tim Brys, Peter Vrancx, Ann Nowe

Recent advances of gradient temporal-difference methods allow to learn off-policy multiple value functions in parallel with- out sacrificing convergence guarantees or computational efficiency. This opens up new possibili…

Computational Efficiency reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Robust and Adaptive Temporal-Difference Learning Using An Ensemble of Gaussian Processes

2021-12-01 · Qin Lu, Georgios B. Giannakis

Value function approximation is a crucial module for policy evaluation in reinforcement learning when the state space is large or continuous. The present paper takes a generative perspective on policy evaluation via temp…

Gaussian Processes

Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks

2022-09-16 · Litian Liang, Yaosheng Xu, Stephen Mcaleer, Dailin Hu 외

In temporal-difference reinforcement learning algorithms, variance in value estimation can cause instability and overestimation of the maximal target value. Many algorithms have been proposed to reduce overestimation, in…

Online Weighted Q-Ensembles for Reduced Hyperparameter Tuning in Reinforcement Learning

2022-09-29 · Renata Garcia, Wouter Caarls

Reinforcement learning is a promising paradigm for learning robot control, allowing complex control policies to be learned without requiring a dynamics model. However, even state of the art algorithms can be difficult to…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Generalized Kalman filter based temporal difference reinforcement learning

2026-07-22 · Vasos Arnaoutis, Eric Lutters, Bojana Rosić arxiv

In this paper, we present a generalized temporal-difference (TD) reinforcement learning framework based on the theory of conditional expectations. The value and action-value (Q-value) functions are treated as uncertain q…

Reinforcement Learning