VisPlay: Self-Evolving Vision-Language Models from Images

2025-11-19 · Yicheng He, Chengsong Huang, Zongxia Li, Jiaxin Huang, Yonghui Yang arxiv

Reinforcement learning (RL) provides a principled framework for improving Vision-Language Models (VLMs) on complex reasoning tasks. However, existing RL approaches often rely on human-annotated labels or task-specific heuristics to define verifiable rewards, both of which are costly and difficult to scale. We introduce VisPlay, a self-evolving RL framework that enables VLMs to autonomously improve their reasoning abilities using large amounts of unlabeled image data. Starting from a single base VLM, VisPlay assigns the model into two interacting roles: an Image-Conditioned Questioner that formulates challenging yet answerable visual questions, and a Multimodal Reasoner that generates silver responses. These roles are jointly trained with Group Relative Policy Optimization (GRPO), which incorporates diversity and difficulty rewards to balance the complexity of generated questions with the quality of the silver answers. VisPlay scales efficiently across two model families. When trained on Qwen2.5-VL and MiMo-VL, VisPlay achieves consistent improvements in visual reasoning, compositional generalization, and hallucination reduction across eight benchmarks, including MM-Vet and MMMU, demonstrating a scalable path toward self-evolving multimodal intelligence. The project page is available at https://bruno686.github.io/VisPlay/

📄 PDF Abstract BibTeX arXiv:2511.15661

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Visual Reasoning

Active Zero: Self-Evolving Vision-Language Models through Active Environment Exploration

2026-02-11 · Jinghan He, Junfeng Fang, Feng Xiong, Zijun Yao 외 arxiv

Self-play has enabled large language models to autonomously improve through self-generated challenges. However, existing self-play methods for vision-language models rely on passive interaction with static image collecti…

MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data

2026-03-10 · Zongxia Li, Hongyang Du, Chengsong Huang, Xiyang Wu 외 arxiv

Self-evolving has emerged as a key paradigm for improving foundational models such as Large Language Models (LLMs) and Vision Language Models (VLMs) with minimal human intervention. While recent approaches have demonstra…

Multimodal Reasoning

RISE: Reliable Improvement in Self-Evolving Vision-Language Models

2026-05-20 · Chaoran Xu, Yingmao Miao, Pengfei Zhang, Hao Dou 외 arxiv

Vision-language models (VLMs) have achieved strong multimodal reasoning capabilities, but further improving them still relies heavily on large-scale human-constructed supervision for post-training. Such supervision is co…

Multimodal Reasoning Question Generation

Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models

2026-06-25 · Shravan Venkatraman, Ritesh Thawkar, Omkar Thawakar, Rao Muhammad Anwer 외 arxiv

Recently, self-evolving large multimodal models (LMMs) have received attention for improving visual reasoning in a purely unsupervised setting. However, multi-role self-play and self-consistency reward schemes in existin…

Visual Question Answering Image Captioning Visual Reasoning

Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning

2025-12-07 · Tingyu Li, Zheng Sun, Jingxuan Wei, Siyuan Li 외 arxiv

Recent vision-language models (VLMs) achieve remarkable reasoning through reinforcement learning (RL), which provides a feasible solution for realizing continuous self-evolving large vision-language models (LVLMs) in the…

Reinforcement Learning