TRON: Targeted Rule-Verifiable Online Environments for Visual Reasoning RL

2026-06-01 · Tianze Yang, Yucheng Shi, Ruitong Sun, Jingyuan Huang, Ninghao Liu, Jin Sun arxiv

Reinforcement learning (RL) for visual reasoning needs scalable, verifiable, and controllable training signals. Existing visual RL post-training trains on static curated datasets, with fixed image-question-answer samples bounded by their collection budget. In this work, we introduce TRON (Targeted, Rule-verifiable Online eNvironments), an online environment substrate: a training rollout is generated on demand by a controllable generator-verifier program that samples a fresh latent visual state, renders an image, asks a question, and exactly verifies the answer. A single run can therefore draw an unbounded stream of fresh instances at the difficulty level required by the current curriculum. The current TRON suite contains 520 environments organized into five ability buckets (spatial, mathematical, diagram, pattern/logic, and counting); the same substrate supports both a single full model trained on all buckets and per-bucket ability-specialist models, with no additional data collection. We also introduce a substrate analysis covering generation reliability, instance and level diversity, cross-environment near-duplicates, and base-model pass rate by difficulty level. RL post-training with METHOD consistently improves performance on ten external multimodal reasoning benchmarks across Qwen3-VL-4B, Qwen2.5-VL-7B, and MiMo-VL-7B-SFT.

📄 PDF Abstract BibTeX arXiv:2606.01599

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Multimodal Reasoning Visual Reasoning

BitsAI-Fix: LLM-Driven Approach for Automated Lint Error Resolution in Practice

2025-08-05 · Yuanpeng Li, Qi Long, Zhiyuan Yao, Jian Xu 외 arxiv

As enterprise codebases continue to grow in scale and complexity, the volume of lint errors far exceeds engineers' manual remediation capacity, leading to continuous accumulation of technical debt and hindered developmen…

Reinforcement Learning

AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines

2026-02-15 · Yifan Wu, Yiran Peng, Yiyu Chen, Jianhao Ruan 외 arxiv

The performance of autonomous Web GUI agents heavily relies on the quality and quantity of their training data. However, a fundamental bottleneck persists: collecting interaction trajectories from real-world websites is …

ASTRA: Automated Synthesis of agentic Trajectories and Reinforcement Arenas

2026-01-29 · Xiaoyu Tian, Haotian Wang, Shuaiting Chen, Hao Zhou 외 arxiv

Large language models (LLMs) are increasingly used as tool-augmented agents for multi-step decision making, yet training robust tool-using agents remains challenging. Existing methods still require manual intervention, d…

Reinforcement Learning Decision Making

Bridging Offline and Online Reinforcement Learning for LLMs

2025-06-26 · Jack Lanchantin, Angelica Chen, Janice Lan, Xian Li 외

We investigate the effectiveness of reinforcement learning methods for finetuning large language models when transitioning from offline to semi-online to fully online regimes for both verifiable and non-verifiable tasks.…

Instruction Following Math reinforcement-learning Reinforcement Learning

RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments

2025-11-10 · Zhiyuan Zeng, Hamish Ivison, Yiping Wang, Lifan Yuan 외 arxiv

We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scal…