paper-with-me

Papers

Generative World Renderer

2026-04-02 · Zheng-Hui Huang, Zhixiang Wang, Jiaming Tan, Ruihan Yu, Yidan Zhang, Bo Zheng, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang arxiv

Scaling generative inverse and forward rendering to real-world scenarios is bottlenecked by the limited realism and temporal coherence of existing synthetic datasets. To bridge this persistent domain gap, we introduce a large-scale, dynamic dataset curated from visually complex AAA games. Using a novel dual-screen stitched capture method, we extracted 4M continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels across diverse scenes, visual effects, and environments, including adverse weather and motion-blur variants. This dataset uniquely advances bidirectional rendering: enabling robust in-the-wild geometry and material decomposition, and facilitating high-fidelity G-buffer-guided video generation. Furthermore, to evaluate the real-world performance of inverse rendering without ground truth, we propose a novel VLM-based assessment protocol measuring semantic, spatial, and temporal consistency. Experiments demonstrate that inverse renderers fine-tuned on our data achieve superior cross-dataset generalization and controllable generation, while our VLM evaluation strongly correlates with human judgment. Combined with our toolkit, our forward renderer enables users to edit styles of AAA games from G-buffers using text prompts.

📄 PDF Abstract BibTeX arXiv:2604.02329

Code (0)

등록된 구현이 없습니다.

Tasks

Inverse RenderingVideo Generation

Similar Papers 제목 키워드 기반

Generative World Renderer at the Speed of Play

2026-07-21 · Guixu Lin, Zheng-Hui Huang, Siqi Yang, Ming-Hsuan Yang 외 hf

Generative world renderer AlayaRenderer receives structured world states exported from physics engines and synthesizes RGB frames. Unlike models that generate frames from text/control-hints prompts, AlayaRenderer preserv…

Inverting Generative Adversarial Renderer for Face Reconstruction

2021-05-06 · CVPR 2021 1 · Jingtan Piao, Keqiang Sun, KwanYee Lin, Quan Wang 외

Given a monocular face image as input, 3D face geometry reconstruction aims to recover a corresponding 3D face mesh. Recently, both optimization-based and learning-based face reconstruction methods have taken advantage o…

Face Reconstruction

Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs

2013-06-29 · NeurIPS 2013 12 · Vikash K. Mansinghka, Tejas D. Kulkarni, Yura N. Perov, Joshua B. Tenenbaum

The idea of computer vision as the Bayesian inverse problem to computer graphics has a long history and an appealing elegance, but it has proved difficult to directly implement. Instead, most vision tasks are approached …

Probabilistic Programming

Video Models as Native 4D Renderers: World-Grounded Conditioning from Animated Mesh

2026-07-30 · Junhao Chen, Mingjin Chen, Henghaofan Zhang, Minglin Chen 외 arxiv

Pretrained video diffusion models can act as renderers when the desired scene state is already specified by an animated mesh, a camera trajectory, and a reference image. This 4D generative rendering setting raises a repr…

Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data

2020-02-28 · Sebastian Lunz, Yingzhen Li, Andrew Fitzgibbon, Nate Kushman

Recent work has shown the ability to learn generative models for 3D shapes from only unstructured 2D images. However, training such models requires differentiating through the rasterization step of the rendering process,…