Rethinking Image-to-3D Generation with Sparse Queries: Efficiency, Capacity, and Input-View Bias
We present SparseGen, a novel framework for efficient image-to-3D generation, which exhibits low input-view bias while being significantly faster. Unlike traditional approaches that rely on dense volumetric grids, triplanes, or pixel-aligned primitives, we model scenes with a compact sparse set of learned 3D anchor queries and a learned expansion operator that decodes each transformed query into a small local set of 3D Gaussian primitives. Trained under a rectified-flow reconstruction objective without 3D supervision, our model learns to allocate representation capacity where geometry and appearance matter, achieving significant reductions in memory and inference time while preserving multi-view fidelity. We introduce quantitative measures of input-view bias and utilization to show that sparse queries reduce overfitting to conditioning views while being representationally efficient. Our results argue that sparse set-latent expansion is a principled, practical alternative for efficient 3D generative modeling.
Code (0)
등록된 구현이 없습니다.
Tasks
3D GenerationSimilar Papers 제목 키워드 기반
Rethinking Sparse Lexical Representations for Image Retrieval in the Age of Rising Multi-Modal Large Language Models
In this paper, we rethink sparse lexical representations for image retrieval. By utilizing multi-modal large language models (M-LLMs) that support visual prompting, we can extract image features and convert them into tex…
Data AugmentationImage RetrievalLanguage ModelingLanguage Modelling+2Cross-modal RAG: Sub-dimensional Retrieval-Augmented Text-to-Image Generation
Text-to-image generation increasingly demands access to domain-specific, fine-grained, and rapidly evolving knowledge that pretrained models cannot fully capture. Existing Retrieval-Augmented Generation (RAG) methods att…
Image GenerationLanguage ModelingLanguage ModellingLarge Language Model+6LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial Attack
We propose LSDAT, an image-agnostic decision-based black-box attack that exploits low-rank and sparse decomposition (LSD) to dramatically reduce the number of queries and achieve superior fooling rates compared to the st…
Adversarial AttackComputational EfficiencyDimensionality ReductionGrid Partitioned Attention: Efficient TransformerApproximation with Inductive Bias for High Resolution Detail Generation
Attention is a general reasoning mechanism than can flexibly deal with image information, but its memory requirements had made it so far impractical for high resolution image generation. We present Grid Partitioned Atten…
Conditional Image GenerationDeep AttentionImage GenerationInductive BiasSGTR: Generating Scene Graph by Learning Compositional Triplets with Transformer
In this work, we propose an end-to-end framework for the scene graph generation. Motivated by the recently introduced DETR, our method, termed SGTR, generating scene graphs by learning compositional queries with Transfor…
Graph GenerationScene Graph GenerationTriplet