online and lightweight kernel-based approximated policy iteration for dynamic p-norm linear adaptive filtering
This paper introduces a solution to the problem of selecting dynamically (online) the `optimal'' p-norm to combat outliers in linear adaptive filtering without any knowledge on the probability density function of the outliers. The proposed online and data-driven framework is built on kernel-based reinforcement learning (KBRL). To this end, novel Bellman mappings on reproducing kernel Hilbert spaces (RKHSs) are introduced. These mappings do not require any knowledge on transition probabilities of Markov decision processes, and are nonexpansive with respect to the underlying Hilbertian norm. The fixed-point sets of the proposed Bellman mappings are utilized to build an approximate policy-iteration (API) framework for the problem at hand. To address the `curse of dimensionality'' in RKHSs, random Fourier features are utilized to bound the computational complexity of the API. Numerical tests on synthetic data for several outlier scenarios demonstrate the superior performance of the proposed API framework over several non-RL and KBRL schemes.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Stability of the Stochastic Gradient Method for an Approximated Large Scale Kernel Machine
In this paper we measured the stability of stochastic gradient method (SGM) for learning an approximated Fourier primal support vector machine. The stability of an algorithm is considered by measuring the generalization …
Binary ClassificationGeneral ClassificationKernel Taylor-Based Value Function Approximation for Continuous-State Markov Decision Processes
We propose a principled kernel-based policy iteration algorithm to solve the continuous-state Markov Decision Processes (MDPs). In contrast to most decision-theoretic planning frameworks, which assume fully known state t…
Approximated Multi-Agent Fitted Q Iteration
We formulate an efficient approximation for multi-agent batch reinforcement learning, the approximated multi-agent fitted Q iteration (AMAFQI). We present a detailed derivation of our approach. We propose an iterative po…
Decision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Feature-Based Aggregation and Deep Reinforcement Learning: A Survey and Some New Implementations
In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcem…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Lambda-Policy Iteration with Randomization for Contractive Models with Infinite Policies: Well-Posedness and Convergence
Abstract dynamic programming models are used to analyze $\lambda$-policy iteration with randomization algorithms. Particularly, contractive models with infinite policies are considered and it is shown that well-posedness…