LayoutTransformer: Relation-Aware Scene Layout Generation
In the areas of machine learning and computer vision, text-to-image synthesis aims at producing image outputs given the input text. In particular, the task of layout generation requires one to describe the spatial information for each object component, with the ability to model their relationships. In this paper, we present a LayoutTransformer Network (LT-Net), which is a generative model for text-conditioned layout generation. By extracting semantics-aware yet object discriminative contextual features from the input, we utilize Gaussian mixture models to describe the layouts for each object with relation consistency enforced. Finally, a co-attention mechanism across textual and visual features is deployed to produce the final output. In our experiments, we conduct extensive experiments on both MS-COCO and Visual Genome (VG) datasets, and confirm the effectiveness and superiority of our LT-Net over recent text-to-image and layout generation models.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationLayout GenerationObjectRelationSimilar Papers 제목 키워드 기반
LayoutTransformer: Scene Layout Generation With Conceptual and Spatial Diversity
When translating text inputs into layouts or images, existing works typically require explicit descriptions of each object in a scene, including their spatial information or the associated relationships. To better ex…
DecoderDiversityLayout GenerationLayoutTransformer: Layout Generation and Completion with Self-attention
We address the problem of scene layout generation for diverse domains such as images, mobile applications, documents, and 3D objects. Most complex scenes, natural or human-designed, can be expressed as a meaningful arran…
Layout GenerationSpatial Information Integration in Small Language Models for Document Layout Generation and Classification
Document layout understanding is a field of study that analyzes the spatial arrangement of information in a document hoping to understand its structure and layout. Models such as LayoutLM (and its subsequent iterations) …
Layout Generationtext-classificationText ClassificationLAW-Diffusion: Complex Scene Generation by Diffusion with Layouts
Thanks to the rapid development of diffusion models, unprecedented progress has been witnessed in image synthesis. Prior works mostly rely on pre-trained linguistic models, but a text is often too abstract to properly sp…
Image GenerationLayout-to-Image GenerationObjectScene GenerationLa La LiDAR: Large-Scale Layout Generation from LiDAR Data
Controllable generation of realistic LiDAR scenes is crucial for applications such as autonomous driving and robotics. While recent diffusion-based models achieve high-fidelity LiDAR generation, they lack explicit contro…
Autonomous DrivingScene Generation