paper-with-me

홈 › Papers

OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward

2026-04-07 · Haoyue Yang, Xuanle Zhao, Xuexin Liu, Feibang Jiang, Yao Zhu arxiv

The paradigm of programmable diagram generation is evolving rapidly, playing a crucial role in structured visualization. However, most existing studies are confined to a narrow range of task formulations and language support, constraining their applicability to diverse diagram types. In this work, we propose OmniDiagram, a unified framework that incorporates diverse diagram code languages and task definitions. To address the challenge of aligning code logic with visual fidelity in Reinforcement Learning (RL), we introduce a novel visual feedback strategy named Visual Interrogation Verifies All (\textsc{Viva}). Unlike brittle syntax-based rules or pixel-level matching, \textsc{Viva} rewards the visual structure of rendered diagrams through a generative approach. Specifically, \textsc{Viva} actively generates targeted visual inquiries to scrutinize diagram visual fidelity and provides fine-grained feedback for optimization. This mechanism facilitates a self-evolving training process, effectively obviating the need for manually annotated ground truth code. Furthermore, we construct M3$^2$Diagram, the first large-scale diagram code generation dataset, containing over 196k high-quality instances. Experimental results confirm that the combination of SFT and our \textsc{Viva}-based RL allows OmniDiagram to establish a new state-of-the-art (SOTA) across diagram code generation benchmarks.

📄 PDF Abstract BibTeX arXiv:2604.05514

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement LearningCode Generation

Similar Papers 제목 키워드 기반

Unified Modeling Language Code Generation from Diagram Images Using Multimodal Large Language Models

2025-03-15 · Averi Bates, Ryan Vavricka, Shane Carleton, Ruosi Shao 외

The Unified Modeling Language is a standardized visual language widely used for modeling and documenting the design of software systems. Although many tools generate UML diagrams from UML code, generating executable UML …

Code GenerationLanguage ModelingLanguage ModellingSSIM

VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing

2026-05-15 · Xiaoyan Su, Peijie Dong, Zhenheng Tang, Song Tang 외 arxiv

Despite the rapid advancements in Vision-Language Models (VLMs), a critical gap remains in their ability to handle structured, controllable diagrammatic tasks essential for professional workflows. Existing methods predom…

Assessing GPT-4-Vision's Capabilities in UML-Based Code Generation

2024-04-22 · Gábor Antal, Richárd Vozár, Rudolf Ferenc

The emergence of advanced neural networks has opened up new ways in automated code generation from conceptual models, promising to enhance software development processes. This paper presents a preliminary evaluation of G…

Code Generation

Unified Architecture Metamodel of Information Systems Developed by Generative AI

2026-03-31 · Oleg Grynets, Vasyl Lyashkevych arxiv

The rapid development of AI and LLMs has driven new methods of SDLC, in which a large portion of code, technical, and business documentation is generated automatically. However, since there is no single architectural fra…

Draw with Thought: Unleashing Multimodal Reasoning for Scientific Diagram Generation

2025-04-13 · Zhiqing Cui, Jiahao Yuan, Hanqing Wang, Yanshu Li 외

Scientific diagrams are vital tools for communicating structured knowledge across disciplines. However, they are often published as static raster images, losing symbolic semantics and limiting reuse. While Multimodal Lar…

Code GenerationMultimodal Reasoningvalid