Basis refinement strategies for linear value function approximation in MDPs
We provide a theoretical framework for analyzing basis function construction for linear value function approximation in Markov Decision Processes (MDPs). We show that important existing methods, such as Krylov bases and Bellman-error-based methods are a special case of the general framework we develop. We provide a general algorithmic framework for computing basis function refinements which “respect” the dynamics of the environment, and we derive approximation error bounds that apply for any algorithm respecting this general framework. We also show how, using ideas related to bisimulation metrics, one can translate basis refinement into a process of finding “prototypes” that are diverse enough to represent the given MDP.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Coupled Depth Learning
In this paper we propose a method for estimating depth from a single image using a coarse to fine approach. We argue that modeling the fine depth details is easier after a coarse depth map has been computed. We express a…
Depth EstimationregressionAdaptive Online Value Function Approximation with Wavelets
Using function approximation to represent a value function is necessary for continuous and high-dimensional state spaces. Linear function approximation has desirable theoretical guarantees and often requires less compute…
AcrobotBasis-to-Basis Operator Learning Using Function Encoders
We present Basis-to-Basis (B2B) operator learning, a novel approach for learning operators on Hilbert spaces of functions based on the foundational ideas of function encoders. We decompose the task of learning operators …
Operator learningTwo-Timescale Networks for Nonlinear Value Function Approximation
A key component for many reinforcement learning agents is to learn a value function, either for policy evaluation or control. Many of the algorithms for learning values, however, are designed for linear function approxim…
Q-LearningReinforcement LearningVocal Bursts Valence PredictionLISR: Learning Linear 3D Implicit Surface Representation Using Compactly Supported Radial Basis Functions
Implicit 3D surface reconstruction of an object from its partial and noisy 3D point cloud scan is the classical geometry processing and 3D computer vision problem. In the literature, various 3D shape representations have…
Surface Reconstruction