홈 › Papers

In-context Reinforcement Learning with Algorithm Distillation

2022-10-25 · Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, Volodymyr Mnih

We propose Algorithm Distillation (AD), a method for distilling reinforcement learning (RL) algorithms into neural networks by modeling their training histories with a causal sequence model. Algorithm Distillation treats learning to reinforcement learn as an across-episode sequential prediction problem. A dataset of learning histories is generated by a source RL algorithm, and then a causal transformer is trained by autoregressively predicting actions given their preceding learning histories as context. Unlike sequential policy prediction architectures that distill post-learning or expert sequences, AD is able to improve its policy entirely in-context without updating its network parameters. We demonstrate that AD can reinforcement learn in-context in a variety of environments with sparse rewards, combinatorial task structure, and pixel-based observations, and find that AD learns a more data-efficient RL algorithm than the one that generated the source data.

📄 PDF Abstract BibTeX arXiv:2210.14215

Code (3)

codelion/optillm pytorch

corl-team/headless-ad pytorch

zzmtsvv/ad-gta pytorch

Tasks

In-Context Reinforcement Learningreinforcement-learning Reinforcement Learning

Vintix: Action Model via In-Context Reinforcement Learning

2025-01-31 · Andrey Polubarov, Nikita Lyubaykin, Alexander Derevyagin, Ilya Zisman 외

In-Context Reinforcement Learning (ICRL) represents a promising paradigm for developing generalist agents that learn at inference time through trial-and-error interactions, analogous to how large language models adapt co…

Decision Making In-Context Reinforcement Learning reinforcement-learning Reinforcement Learning

Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner

2026-04-06 · Andrei Polubarov, Lyubaykin Nikita, Alexander Derevyagin, Artyom Grishin 외 arxiv

Recent progress in in-context reinforcement learning (ICRL) has demonstrated its potential for training generalist agents that can acquire new tasks directly at inference. Algorithm Distillation (AD) pioneered this parad…

Reinforcement Learning

Scaling Algorithm Distillation for Continuous Control with Mamba

2025-06-16 · Samuel Beaussant, Mehdi Mounsif

Algorithm Distillation (AD) was recently proposed as a new approach to perform In-Context Reinforcement Learning (ICRL) by modeling across-episodic training histories autoregressively with a causal transformer model. How…

continuous-control Continuous Control In-Context Reinforcement Learning Mamba+3

Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining

2023-10-12 · Licong Lin, Yu Bai, Song Mei

Large transformer models pretrained on offline reinforcement learning datasets have demonstrated remarkable in-context reinforcement learning (ICRL) capabilities, where they can make good decisions when prompted with int…

In-Context Reinforcement Learning reinforcement-learning Reinforcement Learning Thompson Sampling

Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planning

2025-02-26 · Jaehyeon Son, Soochan Lee, Gunhee Kim

Recent studies have shown that Transformers can perform in-context reinforcement learning (RL) by imitating existing RL algorithms, enabling sample-efficient adaptation to unseen tasks without parameter updates. However,…

In-Context Reinforcement Learning Reinforcement Learning (RL)