Reinforcement Learning of Implicit and Explicit Control Flow in Instructions

2021-02-25 · Ethan A. Brooks, Janarthanan Rajendran, Richard L. Lewis, Satinder Singh

Learning to flexibly follow task instructions in dynamic environments poses interesting challenges for reinforcement learning agents. We focus here on the problem of learning control flow that deviates from a strict step-by-step execution of instructions -- that is, control flow that may skip forward over parts of the instructions or return backward to previously completed or skipped steps. Demand for such flexible control arises in two fundamental ways: explicitly when control is specified in the instructions themselves (such as conditional branching and looping) and implicitly when stochastic environment dynamics require re-completion of instructions whose effects have been perturbed, or opportunistic skipping of instructions whose effects are already present. We formulate an attention-based architecture that meets these challenges by learning, from task reward only, to flexibly attend to and condition behavior on an internal encoding of the instructions. We test the architecture's ability to learn both explicit and implicit control in two illustrative domains -- one inspired by Minecraft and the other by StarCraft -- and show that the architecture exhibits zero-shot generalization to novel instructions of length greater than those in a training set, at a performance level unmatched by two baseline recurrent architectures and one ablation architecture.

📄 PDF Abstract BibTeX arXiv:2102.13195

Code (0)

등록된 구현이 없습니다.

Tasks

Minecraftreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)StarcraftZero-shot Generalization

Neural Procedural Memory: Empowering LLM Agents with Implicit Activation Steering

2026-06-29 · Chengfeng Zhao, Yuqiao Tan, Shizhu He, Yequan Wang 외 arxiv

While Large Language Models (LLMs) excel as static solvers, transforming them into autonomous agents remains challenging. This transition requires continuous environmental interaction, yet current agents lack the necessa…

ImpRIF: Stronger Implicit Reasoning Leads to Better Complex Instruction Following

2026-02-04 · Yuancheng Yang, Lin Yang, Xu Wang, Chao Tong 외 arxiv

As applications of large language models (LLMs) become increasingly complex, the demand for robust complex instruction following capabilities is growing accordingly. We argue that a thorough understanding of the instruct…

Reinforcement Learning Instruction Following

Reinforcement Learning with Latent Flow

2021-01-06 · NeurIPS 2021 12 · Wenling Shang, Xiaofei Wang, Aravind Srinivas, Aravind Rajeswaran 외

Temporal information is essential to learning effective policies with Reinforcement Learning (RL). However, current state-of-the-art RL algorithms either assume that such information is given as part of the state space o…

Atari Games continuous-control Continuous Control Montezuma's Revenge+5

CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrations

2022-10-18 · Kai Yan, Alexander G. Schwing, Yu-Xiong Wang

Although reinforcement learning has found widespread use in dense reward settings, training autonomous agents with sparse rewards remains challenging. To address this difficulty, prior work has shown promising results wh…

Form reinforcement-learning Reinforcement Learning (RL)Retrieval

AllDayNav: Lifelong Navigation via Real-World Reinforcement Learning

2026-06-09 · Hang Yin, Yinan Liang, Jiazhao Zhang, Jiahang Liu 외 arxiv

Lifelong embodied navigation in dynamic environments requires robots to form persistent scene understanding from fragmentary observations, which remains difficult for existing methods that rely on explicit maps or scene …

Reinforcement Learning Scene Understanding