홈 › Papers

Automaton Distillation: Neuro-Symbolic Transfer Learning for Deep Reinforcement Learning

2023-10-29 · Suraj Singireddy, Precious Nwaorgu, Andre Beckus, Aden McKinney, Chinwendu Enyioha, Sumit Kumar Jha, George K. Atia, Alvaro Velasquez

Reinforcement learning (RL) is a powerful tool for finding optimal policies in sequential decision processes. However, deep RL methods have two weaknesses: collecting the amount of agent experience required for practical RL problems is prohibitively expensive, and the learned policies exhibit poor generalization on tasks outside the training data distribution. To mitigate these issues, we introduce automaton distillation, a form of neuro-symbolic transfer learning in which Q-value estimates from a teacher are distilled into a low-dimensional representation in the form of an automaton. We then propose methods for generating Q-value estimates where symbolic information is extracted from a teacher's Deep Q-Network (DQN). The resulting Q-value estimates are used to bootstrap learning in the target discrete and continuous environment via a modified DQN and Twin-Delayed Deep Deterministic (TD3) loss function, respectively. We demonstrate that automaton distillation decreases the time required to find optimal policies for various decision tasks in new environments, even in a target environment different in structure from the source environment.

📄 PDF Abstract BibTeX arXiv:2310.19137

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Methods 이 논문이 사용한 방법론

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

CADENT: Gated Hybrid Distillation for Sample-Efficient Transfer in Reinforcement Learning

2026-01-28 · Mahyar Alinejad, Yue Wang, George Atia arxiv

Transfer learning promises to reduce the high sample complexity of deep reinforcement learning (RL), yet existing methods struggle with domain shift between source and target environments. Policy distillation provides po…

Reinforcement Learning Continuous Control Transfer Learning

Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT

2025-08-22 · Rushitha Santhoshi Mamidala, Anshuman Chhabra, Ankur Mali arxiv

Prompt-based reasoning strategies such as Chain-of-Thought (CoT) and In-Context Learning (ICL) have become widely used for eliciting reasoning capabilities in large language models (LLMs). However, these methods rely on …

Reading Comprehension

Deriving Rewards for Reinforcement Learning from Symbolic Behaviour Descriptions of Bipedal Walking

2023-12-16 · Daniel Harnack, Christoph Lüth, Lukas Gross, Shivesh Kumar 외

Generating physical movement behaviours from their symbolic description is a long-standing challenge in artificial intelligence (AI) and robotics, requiring insights into numerical optimization methods as well as into fo…

reinforcement-learning Reinforcement Learning

Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories

2026-03-26 · Yiyuan Pan, Xusheng Luo, Hanjiang Hu, Peiqi Yu 외 arxiv

Scaling robot learning to long-horizon tasks remains a formidable challenge. While end-to-end policies often lack the structural priors needed for effective long-term reasoning, traditional neuro-symbolic methods rely he…

Continuous Control

LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks

2026-05-06 · Mahyar Alinejad, Yue Wang, Amrit Singh Bedi, George Atia arxiv

Transfer learning in reinforcement learning (RL) seeks to accelerate learning in new tasks by leveraging knowledge from related sources. Existing neurosymbolic transfer methods, however, typically rely on manually specif…

Reinforcement Learning Transfer Learning