Enhanced-FQL($λ$), an Efficient and Interpretable RL with novel Fuzzy Eligibility Traces and Segmented Experience Replay

2026-01-07 · Mohsen Jalaeian-Farimani, Xiong Xiong, Luca Bascetta arxiv

This paper introduces a fuzzy reinforcement learning framework, Enhanced-FQL($λ$), that integrates novel Fuzzified Eligibility Traces (FET) and Segmented Experience Replay (SER) into fuzzy Q-learning with the Fuzzified Bellman Equation (FBE) for continuous control. The proposed approach employs an interpretable fuzzy rule base instead of complex neural architectures, while maintaining competitive performance through two key innovations: a fuzzified Bellman equation with eligibility traces for stable multi-step credit assignment, and a memory-efficient segment-based experience replay mechanism for enhanced sample efficiency. Theoretical analysis proves the proposed method convergence under standard assumptions. On the Cart--Pole benchmark, Enhanced-FQL($λ$) improves sample efficiency and reduces variance relative to $n$-step fuzzy TD and fuzzy SARSA($λ$), while remaining competitive with the tested DDPG baseline. These results support the proposed framework as an interpretable and computationally compact alternative for moderate-scale continuous control problems.

📄 PDF Abstract BibTeX arXiv:2601.04392

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Continuous Control

Investigating Recurrence and Eligibility Traces in Deep Q-Networks

2017-04-18 · Jean Harb, Doina Precup

Eligibility traces in reinforcement learning are used as a bias-variance trade-off and can often speed up training time by propagating knowledge back over time-steps in a single update. We investigate the use of eligibil…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Adaptive and Multiple Time-scale Eligibility Traces for Online Deep Reinforcement Learning

2020-08-23 · Taisuke Kobayashi

Deep reinforcement learning (DRL) is one promising approach to teaching robots to perform complex tasks. Because methods that directly reuse the stored experience data cannot follow the change of the environment in robot…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Learning From the Past with Cascading Eligibility Traces

2025-06-17 · Tokiniaina Raharison Ralambomihanta, Ivan Anokhin, Roman Pogodin, Samira Ebrahimi Kahou 외

Animals often receive information about errors and rewards after a significant delay. For example, there is typically a delay of tens to hundreds of milliseconds between motor actions and visual feedback. The standard ap…

Beyond Uniform Credit Assignment: Selective Eligibility Traces for RLVR

2026-05-07 · Chaoli Mou, Zhan Zhuang, Xinning Chen, Yu Zhang arxiv

Reinforcement Learning with Verifiable Rewards (RLVR) has become a key approach for improving the reasoning abilities of large language models. However, widely used critic-free algorithms such as Group Relative Policy Op…

Reinforcement Learning

One-shot learning and behavioral eligibility traces in sequential decision making

2019-11-12

In many daily tasks we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning theory sugges…

Decision Making Learning Theory One-Shot Learning Pupil Dilation+4