Learning Sparse Representations in Reinforcement Learning with Sparse Coding

2017-07-26 · Lei Le, Raksha Kumaraswamy, Martha White

A variety of representation learning approaches have been investigated for reinforcement learning; much less attention, however, has been given to investigating the utility of sparse coding. Outside of reinforcement learning, sparse coding representations have been widely used, with non-convex objectives that result in discriminative representations. In this work, we develop a supervised sparse coding objective for policy evaluation. Despite the non-convexity of this objective, we prove that all local minima are global minima, making the approach amenable to simple optimization strategies. We empirically show that it is key to use a supervised objective, rather than the more straightforward unsupervised sparse coding approach. We compare the learned representations to a canonical fixed sparse representation, called tile-coding, demonstrating that the sparse coding representation outperforms a wide variety of tilecoding representations.

📄 PDF Abstract BibTeX arXiv:1707.08316

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Representation Learning

The Utility of Sparse Representations for Control in Reinforcement Learning

2018-11-15 · Vincent Liu, Raksha Kumaraswamy, Lei Le, Martha White

We investigate sparse representations for control in reinforcement learning. While these representations are widely used in computer vision, their prevalence in reinforcement learning is limited to sparse coding where ex…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Predictive Coding for Boosting Deep Reinforcement Learning with Sparse Rewards

2019-12-21 · Xingyu Lu, Stas Tiomkin, Pieter Abbeel

While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge. In this work, we propose an effective reward sh…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Sparse Coding with Multi-Layer Decoders using Variance Regularization

2021-12-16 · Katrina Evtimova, Yann Lecun

Sparse representations of images are useful in many computer vision applications. Sparse coding with an $l_1$ penalty and a learned linear dictionary requires regularization of the dictionary to prevent a collapse in the…

Decoder Denoising

Transfer Sparse Coding for Robust Image Representation

2013-06-01 · CVPR 2013 6 · Mingsheng Long, Guiguang Ding, Jian-Min Wang, Jiaguang Sun 외

Sparse coding learns a set of basis functions such that each input signal can be well approximated by a linear combination of just a few of the bases. It has attracted increasing interest due to its state-of-the-art perf…

A Simple Sparse Denoising Layer for Robust Deep Learning

2021-01-01 · Yueming Lyu, Xingrui Yu, Ivor Tsang

Deep models have achieved great success in many applications. However, vanilla deep models are not well-designed against the input perturbation. In this work, we take an initial step to designing a simple robust layer …

Deep Learning Denoising Dictionary Learning Reinforcement Learning (RL)