Using PCA to Efficiently Represent State Spaces
Reinforcement learning algorithms need to deal with the exponential growth of states and actions when exploring optimal control in high-dimensional spaces. This is known as the curse of dimensionality. By projecting the agent's state onto a low-dimensional manifold, we can represent the state space in a smaller and more efficient representation. By using this representation during learning, the agent can converge to a good policy much faster. We test this approach in the Mario Benchmarking Domain. When using dimensionality reduction in Mario, learning converges much faster to a good policy. But, there is a critical convergence-performance trade-off. By projecting onto a low-dimensional manifold, we are ignoring important data. In this paper, we explore this trade-off of convergence and performance. We find that learning in as few as 4 dimensions (instead of 9), we can improve performance past learning in the full dimensional space at a faster convergence rate.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingDimensionality ReductionReinforcement LearningSimilar Papers 제목 키워드 기반
Growing Action Spaces
In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)StarcraftSome observations on high-dimensional partial differential equations with Barron data
We use explicit representation formulas to show that solutions to certain partial differential equations lie in Barron spaces or multilayer spaces if the PDE data lie in such function spaces. Consequently, these solution…
Vocal Bursts Intensity PredictionInformation-based learning by agents in unbounded state spaces
The idea that animals might use information-driven planning to explore an unknown environment and build an internal model of it has been proposed for quite some time. Recent work has demonstrated that agents using this p…
A Slices Perspective for Incremental Nonparametric Inference in High Dimensional State Spaces
We introduce an innovative method for incremental nonparametric probabilistic inference in high-dimensional state spaces. Our approach leverages \slices from high-dimensional surfaces to efficiently approximate posterior…
Policy Networks with Two-Stage Training for Dialogue Systems
In this paper, we propose to use deep policy networks which are trained with an advantage actor-critic method for statistically optimised dialogue systems. First, we show that, on summary state and action spaces, deep Re…
Deep Reinforcement LearningDialogue State TrackingGaussian ProcessesReinforcement Learning+2