Skill Machines: Temporal Logic Skill Composition in Reinforcement Learning
It is desirable for an agent to be able to solve a rich variety of problems that can be specified through language in the same environment. A popular approach towards obtaining such agents is to reuse skills learned in prior tasks to generalise compositionally to new ones. However, this is a challenging problem due to the curse of dimensionality induced by the combinatorially large number of ways high-level goals can be combined both logically and temporally in language. To address this problem, we propose a framework where an agent first learns a sufficient set of skill primitives to achieve all high-level goals in its environment. The agent can then flexibly compose them both logically and temporally to provably achieve temporal logic specifications in any regular language, such as regular fragments of linear temporal logic. This provides the agent with the ability to map from complex temporal logic task specifications to near-optimal behaviours zero-shot. We demonstrate this experimentally in a tabular setting, as well as in a high-dimensional video game and continuous control environment. Finally, we also demonstrate that the performance of skill machines can be improved with regular off-policy reinforcement learning algorithms when optimal behaviours are desired.
Code (1)
Tasks
continuous-controlContinuous Controlreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Automata Guided Skill Composition
Skills learned through (deep) reinforcement learning often generalizes poorly across tasks and re-training is necessary when presented with a new task. We present a framework that combines techniques in formal methods wi…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Neural Associative Skill Memories for safer robotics and modelling human sensorimotor repertoires
Modern robots face challenges shared by humans, where machines must learn multiple sensorimotor skills and express them adaptively. Equipping robots with a human-like memory of how it feels to do multiple stereotypical m…
Fault DetectionOntology-Based Skill Description Learning for Flexible Production Systems
The increasing importance of resource-efficient production entails that manufacturing companies have to create a more dynamic production environment, with flexible manufacturing machines and processes. To fully utilize t…
Inductive logic programmingComposing Task-Agnostic Policies with Deep Reinforcement Learning
The composition of elementary behaviors to solve challenging transfer learning problems is one of the key elements in building intelligent machines. To date, there has been plenty of work on learning task-specific polici…
Decision MakingDeep Reinforcement LearningMotion Planningreinforcement-learning+4SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems
Skill-based agent systems tackle complex tasks by composing reusable skills, improving modularity and scalability while introducing a largely unexamined security attack surface. We propose SkillTrojan, a backdoor attack …