HR-TD: A Regularized TD Method to Avoid Over-Generalization
Temporal Difference learning with function approximation has been widely used recently and has led to several successful results. However, compared with the original tabular-based methods, one major drawback of temporal difference learning with neural networks and other function approximators is that they tend to over-generalize across temporally successive states, resulting in slow convergence and even instability. In this work, we propose a novel TD learning method, Hadamard product Regularized TD (HR-TD), that reduces over-generalization and thus leads to faster convergence. This approach can be easily applied to both linear and nonlinear function approximators. HR-TD is evaluated on several linear and nonlinear benchmark domains, where we show improvement in learning behavior and performance.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Decoupling regularization from the action space
Regularized reinforcement learning (RL), particularly the entropy-regularized kind, has gained traction in optimal control and inverse RL. While standard unregularized RL methods remain unaffected by changes in the numbe…
Reinforcement Learning (RL)Regularized Parameter Uncertainty for Improving Generalization in Reinforcement Learning
In order for reinforcement learning (RL) agents to be deployed in real-world environments they must be able to generalize to unseen environments. However RL struggles with out-of-distribution generalization often due…
Out-of-Distribution Generalizationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Regularization, early-stopping and dreaming: a Hopfield-like setup to address generalization and overfitting
In this work we approach attractor neural networks from a machine learning perspective: we look for optimal network parameters by applying a gradient descent over a regularized loss function. Within this framework, the o…
SRDCNN: Strongly Regularized Deep Convolution Neural Network Architecture for Time-series Sensor Signal Classification Tasks
Deep Neural Networks (DNN) have been successfully used to perform classification and regression tasks, particularly in computer vision based applications. Recently, owing to the widespread deployment of Internet of Thing…
ClassificationGeneral ClassificationTime SeriesTime Series Analysis+1Regularized Distance Metric Learning:Theory and Algorithm
In this paper, we examine the generalization error of regularized distance metric learning. We show that with appropriate constraints, the generalization error of regularized distance metric learning could be independent…
Face RecognitionGeneral ClassificationMetric Learning