Sparse-LaViDa: Sparse Multimodal Discrete Diffusion Language Models
Masked Discrete Diffusion Models (MDMs) have achieved strong performance across a wide range of multimodal tasks, including image understanding, generation, and editing. However, their inference speed remains suboptimal due to the need to repeatedly process redundant masked tokens at every sampling step. In this work, we propose Sparse-LaViDa, a novel modeling framework that dynamically truncates unnecessary masked tokens at each inference step to accelerate MDM sampling. To preserve generation quality, we introduce specialized register tokens that serve as compact representations for the truncated tokens. Furthermore, to ensure consistency between training and inference, we design a specialized attention mask that faithfully matches the truncated sampling procedure during training. Built upon the state-of-the-art unified MDM LaViDa-O, Sparse-LaViDa achieves up to a 2x speedup across diverse tasks including text-to-image generation, image editing, and mathematical reasoning, while maintaining generation quality.
Code (0)
등록된 구현이 없습니다.
Tasks
Text-to-Image GenerationMathematical ReasoningImage EditingSimilar Papers 제목 키워드 기반
LaViDa: A Large Diffusion Language Model for Multimodal Understanding
Modern Vision-Language Models (VLMs) can solve a wide range of tasks requiring visual reasoning. In real-world scenarios, desirable properties for VLMs include fast inference and controllable generation (e.g., constraini…
Instruction FollowingLanguage ModelingLanguage ModellingText Infilling+1Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
We propose Lavida-O, a unified Masked Diffusion Model (MDM) for multimodal understanding and generation. Unlike existing multimodal MDMs such as MMaDa and Muddit which only support simple image-level understanding tasks …
Text-to-Image GenerationMultimodal ReasoningImage EditingLaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models
Diffusion language models (dLLMs) recently emerged as a promising alternative to auto-regressive LLMs. The latest works further extended it to multimodal understanding and generation tasks. In this work, we propose LaViD…
Reinforcement LearningImage EditingLLAVIDAL: A Large LAnguage VIsion Model for Daily Activities of Living
Current Large Language Vision Models (LLVMs) trained on web videos perform well in general video understanding but struggle with fine-grained details, complex human-object interactions (HOI), and view-invariant represent…
BenchmarkingHuman-Object Interaction DetectionRepresentation LearningVideo Description+1The Thinking Pixel: Recursive Sparse Reasoning in Multimodal Diffusion Latents
Diffusion models have achieved success in high-fidelity data synthesis, yet their capacity for more complex, structured reasoning like text following tasks remains constrained. While advances in language models have leve…
Text-to-Image Generation