Autonomous Extraction of a Hierarchical Structure of Tasks in Reinforcement Learning, A Sequential Associate Rule Mining Approach

2018-11-17 · Behzad Ghazanfari, Fatemeh Afghah, Matthew E. Taylor

Reinforcement learning (RL) techniques, while often powerful, can suffer from slow learning speeds, particularly in high dimensional spaces. Decomposition of tasks into a hierarchical structure holds the potential to significantly speed up learning, generalization, and transfer learning. However, the current task decomposition techniques often rely on high-level knowledge provided by an expert (e.g. using dynamic Bayesian networks) to extract a hierarchical task structure; which is not necessarily available in autonomous systems. In this paper, we propose a novel method based on Sequential Association Rule Mining that can extract Hierarchical Structure of Tasks in Reinforcement Learning (SARM-HSTRL) in an autonomous manner for both Markov decision processes (MDPs) and factored MDPs. The proposed method leverages association rule mining to discover the causal and temporal relationships among states in different trajectories, and extracts a task hierarchy that captures these relationships among sub-goals as termination conditions of different sub-tasks. We prove that the extracted hierarchical policy offers a hierarchically optimal policy in MDPs and factored MDPs. It should be noted that SARM-HSTRL extracts this hierarchical optimal policy without having dynamic Bayesian networks in scenarios with a single task trajectory and also with multiple tasks' trajectories. Furthermore, it has been theoretically and empirically shown that the extracted hierarchical task structure is consistent with trajectories and provides the most efficient, reliable, and compact structure under appropriate assumptions. The numerical results compare the performance of the proposed SARM-HSTRL method with conventional HRL algorithms in terms of the accuracy in detecting the sub-goals, the validity of the extracted hierarchies, and the speed of learning in several testbeds.

📄 PDF Abstract BibTeX arXiv:1811.08275

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Autonomous Extracting a Hierarchical Structure of Tasks in Reinforcement Learning and Multi-task Reinforcement Learning

2017-09-14 · Behzad Ghazanfari, Matthew E. Taylor

Reinforcement learning (RL), while often powerful, can suffer from slow learning speeds, particularly in high dimensional spaces. The autonomous decomposition of tasks and use of hierarchical methods hold the potential t…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning

2019-11-09 · Zhiqian Qiao, Zachariah Tyree, Priyantha Mudalige, Jeff Schneider 외

In this work, we propose a hierarchical reinforcement learning (HRL) structure which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals. In this hierarchical stru…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical End-to-End Autonomous Driving: Integrating BEV Perception with Deep Reinforcement Learning

2024-09-26 · Siyi Lu, Lei He, Shengbo Eben Li, Yugong Luo 외

End-to-end autonomous driving offers a streamlined alternative to the traditional modular pipeline, integrating perception, prediction, and planning within a single framework. While Deep Reinforcement Learning (DRL) has …

Autonomous Driving Deep Reinforcement Learning Semantic Segmentation

Hierarchical Program-Triggered Reinforcement Learning Agents For Automated Driving

2021-03-25 · Briti Gangopadhyay, Harshit Soora, Pallab Dasgupta

Recent advances in Reinforcement Learning (RL) combined with Deep Learning (DL) have demonstrated impressive performance in complex tasks, including autonomous driving. The use of RL agents in autonomous driving leads to…

Autonomous Driving Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Towards Autonomous Pipeline Inspection with Hierarchical Reinforcement Learning

2021-07-08 · Nicolò Botteghi, Luuk Grefte, Mannes Poel, Beril Sirmacek 외

Inspection and maintenance are two crucial aspects of industrial pipeline plants. While robotics has made tremendous progress in the mechanic design of in-pipe inspection robots, the autonomous control of such robots is …

Autonomous Navigation Deep Reinforcement Learning Hierarchical Reinforcement Learning reinforcement-learning+2