Efficient Sequential Decision Making with Large Language Models
This paper focuses on extending the success of large language models (LLMs) to sequential decision making. Existing efforts either (i) re-train or finetune LLMs for decision making, or (ii) design prompts for pretrained LLMs. The former approach suffers from the computational burden of gradient updates, and the latter approach does not show promising results. In this paper, we propose a new approach that leverages online model selection algorithms to efficiently incorporate LLMs agents into sequential decision making. Statistically, our approach significantly outperforms both traditional decision making algorithms and vanilla LLM agents. Computationally, our approach avoids the need for expensive gradient updates of LLMs, and throughout the decision making process, it requires only a small number of LLM calls. We conduct extensive experiments to verify the effectiveness of our proposed approach. As an example, on a large-scale Amazon dataset, our approach achieves more than a $6$x performance gain over baselines while calling LLMs in only $1.5$\% of the time steps.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingModel SelectionSequential Decision MakingSimilar Papers 제목 키워드 기반
UNO Arena for Evaluating Sequential Decision-Making Capability of Large Language Models
Sequential decision-making refers to algorithms that take into account the dynamics of the environment, where early decisions affect subsequent decisions. With large language models (LLMs) demonstrating powerful capabili…
Decision MakingSequential Decision MakingVerifying Memoryless Sequential Decision-making of Large Language Models
We introduce a tool for rigorous and automated verification of large language model (LLM)- based policies in memoryless sequential decision-making tasks. Given a Markov decision process (MDP) representing the sequential …
Reinforcement LearningLearning to Generalize for Sequential Decision Making
We consider problems of making sequences of decisions to accomplish tasks, interacting via the medium of language. These problems are often tackled with reinforcement learning approaches. We find that these models do not…
Decision MakingImitation LearningNatural Language Understandingreinforcement-learning+3Large Language Models for Sequential Decision-Making: Improving In-Context Learning via Supervised Fine-Tuning
Large language models (LLMs) have shown remarkable in-context learning (ICL) capabilities, yet their potential for sequential decision-making remains underexplored. In this paper, we study the ICL capabilities of LLMs in…
On the Modeling Capabilities of Large Language Models for Sequential Decision Making
Large pretrained models are showing increasingly better performance in reasoning and planning tasks across different modalities, opening the possibility to leverage them for complex sequential decision making problems. I…
Decision MakingDiversityReinforcement Learning (RL)Sequential Decision Making