Guide Your Agent with Adaptive Multimodal Rewards

2023-09-21 · NeurIPS 2023 11

Developing an agent capable of adapting to unseen environments remains a difficult challenge in imitation learning. This work presents Adaptive Return-conditioned Policy (ARP), an efficient framework designed to enhance the agent's generalization ability using natural language task descriptions and pre-trained multimodal encoders. Our key idea is to calculate a similarity between visual observations and natural language instructions in the pre-trained multimodal embedding space (such as CLIP) and use it as a reward signal. We then train a return-conditioned policy using expert demonstrations labeled with multimodal rewards. Because the multimodal rewards provide adaptive signals at each timestep, our ARP effectively mitigates the goal misgeneralization. This results in superior generalization performances even when faced with unseen text instructions, compared to existing text-conditioned policies. To improve the quality of rewards, we also introduce a fine-tuning method for pre-trained multimodal encoders, further enhancing the performance. Video demonstrations and source code are available on the project website: \url{https://sites.google.com/view/2023arp}.

📄 PDF Abstract BibTeX

Code (1)

csmile-1006/arp 공식 구현 jax

The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards

2024-09-24 · Sukai Huang, Shu-Wei Liu, Nir Lipovetzky, Trevor Cohn

While Vision-Language Models (VLMs) are increasingly used to generate reward signals for training embodied agents to follow instructions, our research reveals that agents guided by VLM rewards often underperform compared…

reinforcement-learning Reinforcement Learning

GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning

2026-01-26 · Kaixun Jiang, Yuzheng Wang, Junjie Zhou, Pandeng Li 외 arxiv

We introduce GenAgent, unifying visual understanding and generation through an agentic multimodal model. Unlike unified models that face expensive training costs and understanding-generation trade-offs, GenAgent decouple…

Text-to-Image Generation Reinforcement Learning Multimodal Reasoning

Multimodal Reinforcement Learning with Adaptive Verifier for AI Agents

2025-12-03 · Reuben Tan, Baolin Peng, Zhengyuan Yang, Hao Cheng 외 arxiv

Agentic reasoning models trained with multimodal reinforcement learning (MMRL) have become increasingly capable, yet they are almost universally optimized using sparse, outcome-based rewards computed based on the final a…

Reinforcement Learning Multimodal Reasoning Spatial Reasoning

Why Keep Your Doubts to Yourself? Trading Visual Uncertainties in Multi-Agent Bandit Systems

2026-01-26 · Jusheng Zhang, Yijia Fan, Kaitong Cai, Jing Yang 외 arxiv

Vision-Language Models (VLMs) enable powerful multi-agent systems, but scaling them is economically unsustainable: coordinating heterogeneous agents under information asymmetry often spirals costs. Existing paradigms, su…

Sample Efficiency in Sparse Reinforcement Learning: Or Your Money Back

2020-08-28 · Trevor A. McInroe

Sparse rewards present a difficult problem in reinforcement learning and may be inevitable in certain domains with complex dynamics such as real-world robotics. Hindsight Experience Replay (HER) is a recent replay memory…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)