paper-with-me

Papers

GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting

2024-02-11 · Xiaoyu Zhou, Xingjian Ran, Yajiao Xiong, Jinlin He, Zhiwei Lin, Yongtao Wang, Deqing Sun, Ming-Hsuan Yang

We present GALA3D, generative 3D GAussians with LAyout-guided control, for effective compositional text-to-3D generation. We first utilize large language models (LLMs) to generate the initial layout and introduce a layout-guided 3D Gaussian representation for 3D content generation with adaptive geometric constraints. We then propose an instance-scene compositional optimization mechanism with conditioned diffusion to collaboratively generate realistic 3D scenes with consistent geometry, texture, scale, and accurate interactions among multiple objects while simultaneously adjusting the coarse layout priors extracted from the LLMs to align with the generated scene. Experiments show that GALA3D is a user-friendly, end-to-end framework for state-of-the-art scene-level 3D content generation and controllable editing while ensuring the high fidelity of object-level entities within the scene. The source codes and models will be available at gala3d.github.io.

📄 PDF Abstract BibTeX arXiv:2402.07207

Code (0)

등록된 구현이 없습니다.

Tasks

3D GenerationScene GenerationText to 3D

Methods 이 논문이 사용한 방법론

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

ChartGalaxy: A Dataset for Infographic Chart Understanding and Generation

2025-05-24 · Zhen Li, Yukai Guo, Duan Li, Xinyuan Guo 외

Infographic charts are a powerful medium for communicating abstract data by combining visual elements (e.g., charts, images) with textual information. However, their visual and structural richness poses challenges for la…

BenchmarkingChart UnderstandingCode GenerationMultimodal Reasoning

Toward Scene Graph and Layout Guided Complex 3D Scene Generation

2024-12-29 · Yu-Hsiang Huang, Wei Wang, Sheng-Yu Huang, Yu-Chiang Frank Wang

Recent advancements in object-centric text-to-3D generation have shown impressive results. However, generating complex 3D scenes remains an open challenge due to the intricate relations between objects. Moreover, existin…

3D GenerationScene GenerationText to 3D

LAYOUTDREAMER: Physics-guided Layout for Text-to-3D Compositional Scene Generation

2025-02-04 · Yang Zhou, Zongjin He, Qixuan Li, Chao Wang

Recently, the field of text-guided 3D scene generation has garnered significant attention. High-quality generation that aligns with physical realism and high controllability is crucial for practical 3D scene applications…

3DGSScene GenerationText to 3D

M3DLayout: A Multi-Source Dataset of 3D Indoor Layouts and Structured Descriptions for 3D Generation

2025-09-28 · Yiheng Zhang, Zhuojiang Cai, Mingdao Wang, Meitong Guo 외 arxiv

In text-driven 3D scene generation, object layout serves as a crucial intermediate representation that bridges high-level language instructions with detailed geometric output. It not only provides a structural blueprint …

Scene Generation3D Generation

CoT-lized Diffusion: Let's Reinforce T2I Generation Step-by-step

2025-07-06 · Zheyuan Liu, Munan Ning, Qihui Zhang, Shuo Yang 외

Current text-to-image (T2I) generation models struggle to align spatial composition with the input text, especially in complex scenes. Even layout-based approaches yield suboptimal spatial control, as their generation pr…

DenoisingLarge Language ModelMultimodal Large Language Model