3D scene generation from scene graphs and self-attention
Synthesizing realistic and diverse indoor 3D scene layouts in a controllable fashion opens up applications in simulated navigation and virtual reality. As concise and robust representations of a scene, scene graphs have proven to be well-suited as the semantic control on the generated layout. We present a variant of the conditional variational autoencoder (cVAE) model to synthesize 3D scenes from scene graphs and floor plans. We exploit the properties of self-attention layers to capture high-level relationships between objects in a scene, and use these as the building blocks of our model. Our model, leverages graph transformers to estimate the size, dimension and orientation of the objects in a room while satisfying relationships in the given scene graph. Our experiments shows self-attention layers leads to sparser (7.9x compared to Graphto3D) and more diverse scenes (16%).
Code (1)
Tasks
Scene GenerationSimilar Papers 제목 키워드 기반
Scene Graph Conditioning in Latent Diffusion
Diffusion models excel in image generation but lack detailed semantic control using text prompts. Additional techniques have been developed to address this limitation. However, conditioning diffusion models solely on tex…
Image GenerationExplainable Action Prediction through Self-Supervision on Scene Graphs
This work explores scene graphs as a distilled representation of high-level information for autonomous driving, applied to future driver-action prediction. Given the scarcity and strong imbalance of data samples, we prop…
Autonomous DrivingPredictionLearning Visual Commonsense for Robust Scene Graph Generation
Scene graph generation models understand the scene through object and predicate recognition, but are prone to mistakes due to the challenges of perception in the wild. Perception errors often lead to nonsensical composit…
Graph GenerationScene Graph GenerationScene UnderstandingMIGS: Meta Image Generation from Scene Graphs
Generation of images from scene graphs is a promising direction towards explicit scene generation and manipulation. However, the images generated from the scene graphs lack quality, which in part comes due to high diffic…
DiversityImage GenerationImage Generation from Scene GraphsMeta-Learning+1A Comprehensive Survey of Scene Graphs: Generation and Application
Scene graph is a structured representation of a scene that can clearly express the objects, attributes, and relationships between objects in the scene. As computer vision technology continues to develop, people are no lo…
Image CaptioningQuestion AnsweringRelationship DetectionRetrieval+5