Reward-Predictive Clustering

2022-11-07 · Lucas Lehnert, Michael J. Frank, Michael L. Littman

Recent advances in reinforcement-learning research have demonstrated impressive results in building algorithms that can out-perform humans in complex tasks. Nevertheless, creating reinforcement-learning systems that can build abstractions of their experience to accelerate learning in new contexts still remains an active area of research. Previous work showed that reward-predictive state abstractions fulfill this goal, but have only be applied to tabular settings. Here, we provide a clustering algorithm that enables the application of such state abstractions to deep learning settings, providing compressed representations of an agent's inputs that preserve the ability to predict sequences of reward. A convergence theorem and simulations show that the resulting reward-predictive deep network maximally compresses the agent's inputs, significantly speeding up learning in high dimensional visual control tasks. Furthermore, we present different generalization experiments and analyze under which conditions a pre-trained reward-predictive representation network can be re-used without re-training to accelerate learning -- a form of systematic out-of-distribution transfer.

📄 PDF Abstract BibTeX arXiv:2211.03281

Code (0)

등록된 구현이 없습니다.

Tasks

Clusteringreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Predictive Overlapping Co-Clustering

2014-03-08 · Chandrima Sarkar, Jaideep Srivastava

In the past few years co-clustering has emerged as an important data mining tool for two way data analysis. Co-clustering is more advantageous over traditional one dimensional clustering in many ways such as, ability to …

Clustering Community Detection Recommendation Systems

Predictive Coding for Boosting Deep Reinforcement Learning with Sparse Rewards

2019-12-21 · Xingyu Lu, Stas Tiomkin, Pieter Abbeel

While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge. In this work, we propose an effective reward sh…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-perspective Imbalance-Conscious 6G Beamforming Optimization and Performance

2026-08-13 · Chukwunonso Henry Nwokoye, Blessing Oluchi Iloka, Chikwue V. Umeugoji, Christopher Anene Egemba 외 arxiv

The study presents a systematic machine learning (ML) study of 6G-IoT beamforming optimization (6GBO) using supervised and unsupervised approaches. We compared the predictive power of network, environmental, device, and …

Reinforcement Learning Feature Importance

Predictive Clustering of Vessel Behavior Based on Hierarchical Trajectory Representation

2024-03-13 · Rui Zhang, Hanyue Wu, Zhenzhong Yin, Zhu Xiao 외

Vessel trajectory clustering, which aims to find similar trajectory patterns, has been widely leveraged in overwater applications. Most traditional methods use predefined rules and thresholds to identify discrete vessel …

Clustering Trajectory Clustering

Unsupervised deep clustering for predictive texture pattern discovery in medical images

2020-01-31 · Matthias Perkonigg, Daniel Sobotka, Ahmed Ba-Ssalamah, Georg Langs

Predictive marker patterns in imaging data are a means to quantify disease and progression, but their identification is challenging, if the underlying biology is poorly understood. Here, we present a method to identify p…

Clustering Deep Clustering