PixelDiT: Pixel Diffusion Transformers for Image Generation

2025-11-25 · Yongsheng Yu, Wei Xiong, Weili Nie, Yichen Sheng, Shiqiu Liu, Jiebo Luo arxiv

Latent-space modeling has been the standard for Diffusion Transformers (DiTs). However, it relies on a two-stage pipeline where the pretrained autoencoder introduces lossy reconstruction, leading to error accumulation while hindering joint optimization. To address these issues, we propose PixelDiT, a single-stage, end-to-end model that eliminates the need for the autoencoder and learns the diffusion process directly in the pixel space. PixelDiT adopts a fully transformer-based architecture shaped by a dual-level design: a patch-level DiT that captures global semantics and a pixel-level DiT that refines texture details, enabling efficient training of a pixel-space diffusion model while preserving fine details. PixelDiT achieves 1.61 FID on ImageNet 256 and 1.81 FID on ImageNet 512, surpassing existing pixel generative models. We further extend PixelDiT to text-to-image generation and pretrain it at the 10242resolution in pixel space. It achieves 0.74 on GenEval and 83.5 on DPG-bench, approaching the best latent diffusion models. Code: https://github.com/NVlabs/PixelDiT

📄 PDF Abstract BibTeX arXiv:2511.20645

Code (0)

등록된 구현이 없습니다.

Tasks

Text-to-Image Generation

Three-Body Scattering for Generative Modeling

2026-07-20 · Peng Sun, Zhenglin Cheng, Deyuan Liu, Jun Xie 외 hf

Modern generative models typically rely on an adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional energy can induce sample-level motion a…

Pixel-Space Diffusion Transformers

2026-07-20 · Renye Yan, Jikang Cheng, You Wu, Ling Liang 외 arxiv

Latent diffusion models (LDMs) enable efficient high-resolution image synthesis by denoising in a VAE-compressed latent space. However, fixed visual tokenizers can discard fine textures and structural details, while sepa…

Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers

2025-10-08 · Gangwei Xu, Haotong Lin, Hongcheng Luo, Xianqi Wang 외 arxiv

This paper presents Pixel-Perfect Depth, a monocular depth estimation model based on pixel-space diffusion generation that produces high-quality, flying-pixel-free point clouds from estimated depth maps. Current generati…

Monocular Depth Estimation Point Clouds

An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion

2024-08-06 · Xingguang Yan, Han-Hung Lee, Ziyu Wan, Angel X. Chang

We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pi…

3D Shape Generation Image Generation Object

WiT: Waypoint Diffusion Transformers for Alleviating Trajectory Conflict in Pixel-Space Image Generation

2026-03-16 · Hainuo Wang, Mingjia Li, Xiaojie Guo arxiv

While recent Flow Matching models avoid the reconstruction bottlenecks of latent autoencoders by operating directly in pixel space, the raw pixel manifold provides little explicit semantic organization, making target-spe…