Value function approximation via low-rank models
We propose a novel value function approximation technique for Markov decision processes. We consider the problem of compactly representing the state-action value function using a low-rank and sparse matrix model. The problem is to decompose a matrix that encodes the true value function into low-rank and sparse components, and we achieve this using Robust Principal Component Analysis (PCA). Under minimal assumptions, this Robust PCA problem can be solved exactly via the Principal Component Pursuit convex optimization problem. We experiment the procedure on several examples and demonstrate that our method yields approximations essentially identical to the true function.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Robust Subspace Clustering via Tighter Rank Approximation
Matrix rank minimization problem is in general NP-hard. The nuclear norm is used to substitute the rank function in many recent studies. Nevertheless, the nuclear norm approximation adds all singular values together and …
ClusteringFace ClusteringMotion SegmentationLow-rank State-action Value-function Approximation
Value functions are central to Dynamic Programming and Reinforcement Learning but their exact estimation suffers from the curse of dimensionality, challenging the development of practical value-function (VF) estimation a…
Q-LearningUncertainty-aware Low-Rank Q-Matrix Estimation for Deep Reinforcement Learning
Value estimation is one key problem in Reinforcement Learning. Albeit many successes have been achieved by Deep Reinforcement Learning (DRL) in different fields, the underlying structure and learning dynamics of value fu…
continuous-controlContinuous ControlDeep Reinforcement LearningMuJoCo+3Towards a Zero-One Law for Column Subset Selection
There are a number of approximation algorithms for NP-hard versions of low rank approximation, such as finding a rank-$k$ matrix $B$ minimizing the sum of absolute values of differences to a given $n$-by-$n$ matrix $A$, …
Fast Singular Value Shrinkage with Chebyshev Polynomial Approximation Based on Signal Sparsity
We propose an approximation method for thresholding of singular values using Chebyshev polynomial approximation (CPA). Many signal processing problems require iterative application of singular value decomposition (SVD) f…