Self-Supervised Road Layout Parsing with Graph Auto-Encoding
Aiming for higher-level scene understanding, this work presents a neural network approach that takes a road-layout map in bird's-eye-view as input, and predicts a human-interpretable graph that represents the road's topological layout. Our approach elevates the understanding of road layouts from pixel level to the level of graphs. To achieve this goal, an image-graph-image auto-encoder is utilized. The network is designed to learn to regress the graph representation at its auto-encoder bottleneck. This learning is self-supervised by an image reconstruction loss, without needing any external manual annotations. We create a synthetic dataset containing common road layout patterns and use it for training of the auto-encoder in addition to the real-world Argoverse dataset. By using this additional synthetic dataset, which conceptually captures human knowledge of road layouts and makes this available to the network for training, we are able to stabilize and further improve the performance of topological road layout understanding on the real-world Argoverse dataset. The evaluation shows that our approach exhibits comparable performance to a strong fully-supervised baseline.
Code (1)
Tasks
Image ReconstructionScene UnderstandingSimilar Papers 제목 키워드 기반
Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing
Document parsing from scanned images into structured formats remains a significant challenge due to its complexly intertwined elements such as text paragraphs, figures, formulas, and tables. Existing supervised fine-tuni…
Reinforcement LearningNeural Turtle Graphics for Modeling City Road Layouts
We propose Neural Turtle Graphics (NTG), a novel generative model for spatial graphs, and demonstrate its applications in modeling city road layouts. Specifically, we represent the road layout using a graph where nodes i…
Self-supervised Photographic Image Layout Representation Learning
In the domain of image layout representation learning, the critical process of translating image layouts into succinct vector forms is increasingly significant across diverse applications, such as image retrieval, manipu…
Image RetrievalRepresentation LearningSelf-Supervised LearningLearning a Model for Inferring a Spatial Road Lane Network Graph using Self-Supervision
Interconnected road lanes are a central concept for navigating urban roads. Currently, most autonomous vehicles rely on preconstructed lane maps as designing an algorithmic model is difficult. However, the generation and…
Autonomous VehiclesSelf-Supervised LearningDolphin-v2: Universal Document Parsing via Scalable Anchor Prompting
Document parsing has garnered widespread attention as vision-language models (VLMs) advance OCR capabilities. However, the field remains fragmented across dozens of specialized models with varying strengths, forcing user…
Attribute Extraction