Accelerating Goal-Directed Reinforcement Learning by Model Characterization

2019-01-04 · Shoubhik Debnath, Gaurav Sukhatme, Lantao Liu

We propose a hybrid approach aimed at improving the sample efficiency in goal-directed reinforcement learning. We do this via a two-step mechanism where firstly, we approximate a model from Model-Free reinforcement learning. Then, we leverage this approximate model along with a notion of reachability using Mean First Passage Times to perform Model-Based reinforcement learning. Built on such a novel observation, we design two new algorithms - Mean First Passage Time based Q-Learning (MFPT-Q) and Mean First Passage Time based DYNA (MFPT-DYNA), that have been fundamentally modified from the state-of-the-art reinforcement learning techniques. Preliminary results have shown that our hybrid approaches converge with much fewer iterations than their corresponding state-of-the-art counterparts and therefore requiring much fewer samples and much fewer training trials to converge.

📄 PDF Abstract BibTeX arXiv:1901.01977

Code (0)

등록된 구현이 없습니다.

Tasks

modelModel-based Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Measuring Goal-Directedness

2024-12-06 · Matt MacDermott, James Fox, Francesco Belardinelli, Tom Everitt

We define maximum entropy goal-directedness (MEG), a formal measure of goal-directedness in causal models and Markov decision processes, and give algorithms for computing it. Measuring goal-directedness is important, as …

Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation

2025-09-30 · Yang Zhang, Huiwen Yan, Mushuang Liu arxiv

Model-Agnostic Meta-Learning (MAML) is a versatile meta-learning framework applicable to both supervised learning and reinforcement learning (RL). However, applying MAML to meta-reinforcement learning (meta-RL) presents …

Computational Efficiency Reinforcement Learning

Towards Measuring Goal-Directedness in AI Systems

2024-10-07 · Dylan Xu, Juan-Pablo Rivera

Recent advances in deep learning have brought attention to the possibility of creating advanced, general AI systems that outperform humans across many tasks. However, if these systems pursue unintended goals, there could…

Reinforcement Learning (RL)

Goal-Directed Planning by Reinforcement Learning and Active Inference

2021-06-18 · Dongqi Han, Kenji Doya, Jun Tani

What is the difference between goal-directed and habitual behavior? We propose a novel computational framework of decision making with Bayesian inference, in which everything is integrated as an entire neural network mod…

Bayesian Inference Decision Making reinforcement-learning Reinforcement Learning+1

Learning, Fast and Slow: A Goal-Directed Memory-Based Approach for Dynamic Environments

2023-01-31 · John Chong Min Tan, Mehul Motani

Model-based next state prediction and state value prediction are slow to converge. To address these challenges, we do the following: i) Instead of a neural network, we do model-based planning using a parallel memory retr…

Reinforcement Learning (RL)Retrieval Value prediction