Scene Designer: a Unified Model for Scene Search and Synthesis from Sketch
Scene Designer is a novel method for searching and generating images using free-hand sketches of scene compositions; i.e. drawings that describe both the appearance and relative positions of objects. Our core contribution is a single unified model to learn both a cross-modal search embedding for matching sketched compositions to images, and an object embedding for layout synthesis. We show that a graph neural network (GNN) followed by Transformer under our novel contrastive learning setting is required to allow learning correlations between object type, appearance and arrangement, driving a mask generation module that synthesises coherent scene layouts, whilst also delivering state of the art sketch based visual search of scenes.
Code (1)
Tasks
Contrastive LearningGraph Neural NetworkObjectMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Prompts
Controllability plays a crucial role in the practical applications of 3D indoor scene synthesis. Existing works either allow rough language-based control, that is convenient but lacks fine-grained scene customization, or…
DenoisingIndoor Scene SynthesisScene GenerationHuman Motion Synthesis in 3D Scenes via Unified Scene Semantic Occupancy
Human motion synthesis in 3D scenes relies heavily on scene comprehension, while current methods focus mainly on scene structure but ignore the semantic understanding. In this paper, we propose a human motion synthesis f…
Dimensionality ReductionMotion SynthesisForest2Seq: Revitalizing Order Prior for Sequential Indoor Scene Synthesis
Synthesizing realistic 3D indoor scenes is a challenging task that traditionally relies on manual arrangement and annotation by expert designers. Recent advances in autoregressive models have automated this process, but …
Indoor Scene SynthesisScene GenerationEnd-to-end Generative Floor-plan and Layout with Attributes and Relation Graph
In this paper, we propose an end-end model for producing furniture layout for interior scene synthesis from the random vector. This proposed model is aimed to support professional interior designers to produce the interi…
Layout GenerationRelationAdversarial Model for Rotated Indoor Scenes Planning
In this paper, we propose an adversarial model for producing furniture layout for interior scene synthesis when the interior room is rotated. The proposed model combines a conditional adversarial network, a rotation modu…
Layout Generationmodel