Zero-shot Model-based Reinforcement Learning using Large Language Models
The emerging zero-shot capabilities of Large Language Models (LLMs) have led to their applications in areas extending well beyond natural language processing tasks. In reinforcement learning, while LLMs have been extensively used in text-based environments, their integration with continuous state spaces remains understudied. In this paper, we investigate how pre-trained LLMs can be leveraged to predict in context the dynamics of continuous Markov decision processes. We identify handling multivariate data and incorporating the control signal as key challenges that limit the potential of LLMs' deployment in this setup and propose Disentangled In-Context Learning (DICL) to address them. We present proof-of-concept applications in two reinforcement learning settings: model-based policy evaluation and data-augmented off-policy reinforcement learning, supported by theoretical analysis of the proposed methods. Our experiments further demonstrate that our approach produces well-calibrated uncertainty estimates. We release the code at https://github.com/abenechehab/dicl.
Code (1)
Tasks
In-Context LearningModel-based Reinforcement Learningreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Multimodal Knowledge Alignment with Reinforcement Learning
Large language models readily adapt to novel settings, even without task-specific training data. Can their zero-shot capacity be extended to multimodal inputs? In this work, we propose ESPER which extends language-only z…
Audio captioningLanguage ModelingLanguage Modellingreinforcement-learning+3Zero-Shot Dual Machine Translation
Neural Machine Translation (NMT) systems rely on large amounts of parallel data. This is a major challenge for low-resource languages. Building on recent work on unsupervised and semi-supervised methods, we present an ap…
Machine TranslationNMTReinforcement LearningTranslationTRN-R1-Zero: Text-rich Network Reasoning via LLMs with Reinforcement Learning Only
Zero-shot reasoning on text-rich networks (TRNs) remains a challenging frontier, as models must integrate textual semantics with relational structure without task-specific supervision. While graph neural networks rely on…
Reinforcement LearningRelational ReasoningText-to-SPARQL Generation with Reinforcement Learning: A GRPO-based Approach on DBLP
Knowledge graph question answering seeks to translate natural language questions into executable queries over knowledge graphs, but existing approaches often rely on large models or full supervision in the form of gold q…
Graph Question AnsweringReinforcement LearningKnowledge GraphsZero-Shot Reinforcement Learning from Low Quality Data
Zero-shot reinforcement learning (RL) promises to provide agents that can perform any task in an environment after an offline, reward-free pre-training phase. Methods leveraging successor measures and successor features …
Offline RLreinforcement-learningReinforcement LearningReinforcement Learning (RL)