VLM-Guided Experience Replay

2026-02-02 · Elad Sharony, Tom Jurgenson, Orr Krupnik, Dotan Di Castro, Shie Mannor arxiv

Recent advances in Large Language Models (LLMs) and Vision-Language Models (VLMs) have enabled powerful semantic and multimodal reasoning capabilities, creating new opportunities to enhance sample efficiency, high-level planning, and interpretability in reinforcement learning (RL). While prior work has integrated LLMs and VLMs into various components of RL, the replay buffer, a core component for storing and reusing experiences, remains unexplored. We propose addressing this gap by leveraging VLMs to guide the prioritization of experiences in the replay buffer. Our key idea is to use a frozen, pre-trained VLM (requiring no fine-tuning) as an automated evaluator to identify and prioritize promising sub-trajectories from the agent's experiences. Across scenarios, including game-playing and robotics, spanning both discrete and continuous domains, agents trained with our proposed prioritization method achieve 11-52% higher average success rates and improve sample efficiency by 19-45% compared to previous approaches. https://esharony.me/projects/vlm-rb/

📄 PDF Abstract BibTeX arXiv:2602.01915

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Multimodal Reasoning

Curriculum-guided Hindsight Experience Replay

2019-12-01 · NeurIPS 2019 12 · Meng Fang, Tianyi Zhou, Yali Du, Lei Han 외

In off-policy deep reinforcement learning, it is usually hard to collect sufficient successful experiences with sparse rewards to learn from. Hindsight experience replay (HER) enables an agent to learn from failures by t…

Deep Reinforcement Learning Diversity Reinforcement Learning

Saliency Guided Experience Packing for Replay in Continual Learning

2021-09-10 · Gobinda Saha, Kaushik Roy

Artificial learning systems aspire to mimic human intelligence by continually learning from a stream of tasks without forgetting past knowledge. One way to enable such learning is to store past experiences in the form of…

Continual Learning Image Classification

Deep In-GPU Experience Replay

2018-01-09 · Ben Parr

Experience replay allows a reinforcement learning agent to train on samples from a large amount of the most recent experiences. A simple in-RAM experience replay stores these most recent experiences in a list in RAM, and…

CPU GPU Reinforcement Learning

Advances in Experience Replay

2018-05-15 · Tracy Wan, Neil Xu

This project combines recent advances in experience replay techniques, namely, Combined Experience Replay (CER), Prioritized Experience Replay (PER), and Hindsight Experience Replay (HER). We show the results of combinat…

OpenAI Gym

Prioritized Experience Replay

2015-11-18 · Tom Schaul, John Quan, Ioannis Antonoglou, David Silver

Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory. However, this approach simply r…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)