Co-Reinforcement Learning for Unified Multimodal Understanding and Generation

2025-05-23 · Jingjing Jiang, Chongjie Si, Jun Luo, Hanwang Zhang, Chao Ma

This paper presents a pioneering exploration of reinforcement learning (RL) via group relative policy optimization for unified multimodal large language models (ULMs), aimed at simultaneously reinforcing generation and understanding capabilities. Through systematic pilot studies, we uncover the significant potential of ULMs to enable the synergistic co-evolution of dual capabilities within a shared policy optimization framework. Building on this insight, we introduce CoRL, a co-reinforcement learning framework comprising a unified RL stage for joint optimization and a refined RL stage for task-specific enhancement. With the proposed CoRL, our resulting model, ULM-R1, achieves average improvements of 7% on three text-to-image generation datasets and 23% on nine multimodal understanding benchmarks. These results demonstrate the effectiveness of CoRL and highlight the substantial benefit of reinforcement learning in facilitating cross-task synergy and optimization for ULMs. Code is available at https://github.com/mm-vl/ULM-R1.

📄 PDF Abstract BibTeX arXiv:2505.17534

Code (1)

mm-vl/ulm-r1 공식 구현 pytorch

Tasks

Image Generation reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Text to Image Generation Text-to-Image Generation

UniRL-Zero: Reinforcement Learning on Unified Models with Joint Language Model and Diffusion Model Experts

2025-10-20 · Fu-Yun Wang, Han Zhang, Michael Gharbi, Hongsheng Li 외 arxiv

We present UniRL-Zero, a unified reinforcement learning (RL) framework that boosts, multimodal language model understanding and reasoning, diffusion model multimedia generation, and their beneficial interaction capabilit…

Reinforcement Learning

LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models

2026-02-15 · Shufan Li, Yuchen Zhu, Jiuxiang Gu, Kangning Liu 외 arxiv

Diffusion language models (dLLMs) recently emerged as a promising alternative to auto-regressive LLMs. The latest works further extended it to multimodal understanding and generation tasks. In this work, we propose LaViD…

Reinforcement Learning Image Editing

LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generation

2026-03-29 · Shentong Mo, Sukmin Yun arxiv

Unified multimodal pretraining has emerged as a promising paradigm for jointly modeling language and vision within a single foundation model. However, existing approaches largely rely on implicit or indirect alignment si…

Visual Reasoning

Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization

2026-03-10 · Ming Nie, Chunwei Wang, Jianhua Han, Hang Xu 외 arxiv

Unified vision-language models have made significant progress in multimodal understanding and generation, yet they largely fall short in producing multimodal interleaved outputs, which is a crucial capability for tasks l…

Text-to-Image Generation Reinforcement Learning Visual Storytelling Visual Reasoning

UnifiedVisual: A Framework for Constructing Unified Vision-Language Datasets

2025-09-18 · Pengyu Wang, Shaojun Zhou, Chenkun Tan, Xinghao Wang 외 arxiv

Unified vision large language models (VLLMs) have recently achieved impressive advancements in both multimodal understanding and generation, powering applications such as visual question answering and text-guided image s…

Visual Question Answering