Combining imagination and heuristics to learn strategies that generalize

2018-09-10 · Erik J Peterson, Necati Alp Müyesser, Timothy Verstynen, Kyle Dunovan

Deep reinforcement learning can match or exceed human performance in stable contexts, but with minor changes to the environment artificial networks, unlike humans, often cannot adapt. Humans rely on a combination of heuristics to simplify computational load and imagination to extend experiential learning to new and more challenging environments. Motivated by theories of the hierarchical organization of the human prefrontal networks, we have developed a model of hierarchical reinforcement learning that combines both heuristics and imagination into a stumbler-strategist network. We test performance of this network using Wythoff's game, a gridworld environment with a known optimal strategy. We show that a heuristic labeling of each position as hot or cold, combined with imagined play, both accelerates learning and promotes transfer to novel games, while also improving model interpretability.

📄 PDF Abstract BibTeX arXiv:1809.03406

Code (1)

CoAxLab/azad 공식 구현 pytorch

Tasks

Deep Reinforcement LearningHierarchical Reinforcement LearningPositionreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Policy Augmentation: An Exploration Strategy for Faster Convergence of Deep Reinforcement Learning Algorithms

2021-02-10 · Arash Mahyari

Despite advancements in deep reinforcement learning algorithms, developing an effective exploration strategy is still an open problem. Most existing exploration strategies either are based on simple heuristics, or requir…

Deep Reinforcement Learning Matrix Completion reinforcement-learning Reinforcement Learning+1

When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoning

2026-02-09 · Shoubin Yu, Yue Zhang, Zun Wang, Jaehong Yoon 외 arxiv

Despite rapid progress in MLLMs, visual spatial reasoning remains unreliable when correct answers depend on how a scene would appear under unseen or alternative viewpoints. Recent work addresses this by augmenting reason…

Spatial Reasoning

Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics

2021-10-04 · EMNLP (insights) 2021 11 · Prajjwal Bhargava, Aleksandr Drozd, Anna Rogers

Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of…

Imagination-Augmented Agents for Deep Reinforcement Learning

2017-07-19 · NeurIPS 2017 12 · Théophane Weber, Sébastien Racanière, David P. Reichert, Lars Buesing 외

We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing model-based reinforcement learning and pla…

Deep Reinforcement Learning Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Efficient Intrinsically Motivated Robotic Grasping with Learning-Adaptive Imagination in Latent Space

2019-10-10 · Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel, Stefan Wermter

Combining model-based and model-free deep reinforcement learning has shown great promise for improving sample efficiency on complex control tasks while still retaining high performance. Incorporating imagination is a rec…

Deep Reinforcement Learning Reinforcement Learning Robotic Grasping