SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning
Deep reinforcement learning (DRL) has gained great success by learning directly from high-dimensional sensory inputs, yet is notorious for the lack of interpretability. Interpretability of the subtasks is critical in hierarchical decision-making as it increases the transparency of black-box-style DRL approach and helps the RL practitioners to understand the high-level behavior of the system better. In this paper, we introduce symbolic planning into DRL and propose a framework of Symbolic Deep Reinforcement Learning (SDRL) that can handle both high-dimensional sensory inputs and symbolic planning. The task-level interpretability is enabled by relating symbolic actions to options.This framework features a planner -- controller -- meta-controller architecture, which takes charge of subtask scheduling, data-driven subtask learning, and subtask evaluation, respectively. The three components cross-fertilize each other and eventually converge to an optimal symbolic plan along with the learned subtasks, bringing together the advantages of long-term planning capability with symbolic knowledge and end-to-end reinforcement learning directly from a high-dimensional sensory input. Experimental results validate the interpretability of subtasks, along with improved data efficiency compared with state-of-the-art approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingDeep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)SchedulingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multivariate Distributional Reinforcement Learning Using Sliced Divergences
Distributional reinforcement learning (DRL) models the full return distribution rather than expectations, but extending it to multivariate settings remains challenging. Many common metrics do not naturally generalize bey…
Reinforcement LearningAtari GamesPosterior Sampling for Deep Reinforcement Learning
Despite remarkable successes, deep reinforcement learning algorithms remain sample inefficient: they require an enormous amount of trial and error to find good policies. Model-based algorithms promise sample efficiency b…
Computational EfficiencyDeep Reinforcement LearningModel-based Reinforcement Learningreinforcement-learning+3Learning from Self-Debate: Preparing Reasoning Models for Multi-Agent Debate
The reasoning abilities of large language models (LLMs) have been substantially improved by reinforcement learning with verifiable rewards (RLVR). At test time, collaborative reasoning through Multi-Agent Debate (MAD) ha…
Reinforcement LearningShielded Deep Reinforcement Learning for Complex Spacecraft Tasking
Autonomous spacecraft control via Shielded Deep Reinforcement Learning (SDRL) has become a rapidly growing research area. However, the construction of shields and the definition of tasking remains informal, resulting in …
Deep Reinforcement Learningreinforcement-learningReinforcement LearningInterpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programming
Recently deep reinforcement learning has achieved tremendous success in wide ranges of applications. However, it notoriously lacks data-efficiency and interpretability. Data-efficiency is important as interacting with th…
Deep Reinforcement LearningHierarchical Reinforcement LearningInductive logic programmingreinforcement-learning+2