Consolidated Adaptive T-soft Update for Deep Reinforcement Learning

2022-02-25 · Taisuke Kobayashi

Demand for deep reinforcement learning (DRL) is gradually increased to enable robots to perform complex tasks, while DRL is known to be unstable. As a technique to stabilize its learning, a target network that slowly and asymptotically matches a main network is widely employed to generate stable pseudo-supervised signals. Recently, T-soft update has been proposed as a noise-robust update rule for the target network and has contributed to improving the DRL performance. However, the noise robustness of T-soft update is specified by a hyperparameter, which should be tuned for each task, and is deteriorated by a simplified implementation. This study develops adaptive T-soft (AT-soft) update by utilizing the update rule in AdaTerm, which has been developed recently. In addition, the concern that the target network does not asymptotically match the main network is mitigated by a new consolidation for bringing the main network back to the target network. This so-called consolidated AT-soft (CAT-soft) update is verified through numerical simulations.

📄 PDF Abstract BibTeX arXiv:2202.12504

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

t-Soft Update of Target Network for Deep Reinforcement Learning

2020-08-25 · Taisuke Kobayashi, Wendyam Eric Lionel Ilboudo

This paper proposes a new robust update rule of target network for deep reinforcement learning (DRL), to replace the conventional update rule, given as an exponential moving average. The target network is for smoothly ge…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Soft Adaptive Policy Optimization

2025-11-25 · Chang Gao, Chujie Zheng, Xiong-Hui Chen, Kai Dang 외 arxiv

Reinforcement learning (RL) plays an increasingly important role in enhancing the reasoning capabilities of large language models (LLMs), yet stable and performant policy optimization remains challenging. Token-level imp…

Reinforcement Learning Mathematical Reasoning

Useful Memories Become Faulty When Continuously Updated by LLMs

2026-05-13 · Dylan Zhang, Yanshan Lin, Zhengkun Wu, Yihang Sun 외 arxiv

Learning from past experience benefits from two complementary forms of memory: episodic traces -- raw trajectories of what happened -- and consolidated abstractions distilled across many episodes into reusable, schema-li…

Reinforcement Learning with Dynamic Boltzmann Softmax Updates

2019-03-14 · Ling Pan, Qingpeng Cai, Qi Meng, Wei Chen 외

Value function estimation is an important task in reinforcement learning, i.e., prediction. The Boltzmann softmax operator is a natural value estimator and can provide several benefits. However, it does not satisfy the n…

Atari Games Q-Learning reinforcement-learning Reinforcement Learning+1

SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution

2025-07-31 · Han Li, Yuling Shi, Shaoxin Lin, Xiaodong Gu 외 arxiv

Issue resolution has made remarkable progress thanks to the advanced reasoning capabilities of large language models (LLMs). Recently, agent-based frameworks such as SWE-agent have further advanced this progress by enabl…