RealGraph: A Multiview Dataset for 4D Real-world Context Graph Generation
In this paper, we propose a brand new scene understanding paradigm called "Context Graph Generation (CGG)", aiming at abstracting holistic semantic information in the complicated 4D world. The CGG task capitalizes on the calibrated multiview videos of a dynamic scene, and targets at recovering semantic information (coordination, trajectories and relationships) of the presented objects in the form of spatio-temporal context graph in 4D space. We also present a benchmark 4D video dataset "RealGraph", the first dataset tailored for the proposed CGG task. The raw data of RealGraph is composed of calibrated and synchronized multiview videos. We exclusively provide manual annotations including object 2D&3D bounding boxes, category labels and semantic relationships. We also make sure the annotated ID for every single object is temporally and spatially consistent. We propose the first CGG baseline algorithm, Multiview-based Context Graph Generation Network (MCGNet), to empirically investigate the legitimacy of CGG task on RealGraph dataset. We nevertheless reveal the great challenges behind this task and encourage the community to explore beyond our solution.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph GenerationScene UnderstandingSimilar Papers 제목 키워드 기반
EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups
Recent advancements in neural surface reconstruction have significantly enhanced 3D reconstruction. However, current real world datasets mainly focus on benchmarking multiview stereo (MVS) based on RGB inputs. Multiview …
3D ReconstructionDriving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving
In autonomous driving, predicting future events in advance and evaluating the foreseeable risks empowers autonomous vehicles to better plan their actions, enhancing safety and efficiency on the road. To this end, we prop…
Autonomous DrivingAutonomous VehiclesUsing Multiple Dermoscopic Photographs of One Lesion Improves Melanoma Classification via Deep Learning: A Prognostic Diagnostic Accuracy Study
Background: Convolutional neural network (CNN)-based melanoma classifiers face several challenges that limit their usefulness in clinical practice. Objective: To investigate the impact of multiple real-world dermoscopic …
DiagnosticEquiNMF: Graph Regularized Multiview Nonnegative Matrix Factorization
Nonnegative matrix factorization (NMF) methods have proved to be powerful across a wide range of real-world clustering applications. Integrating multiple types of measurements for the same objects/subjects allows us to g…
ClusteringData IntegrationDCVC-MV: Deep Contextual Multiview Video Compression with Efficient Inter-View Prediction
Multiview video is a key format for 3D applications such as free-viewpoint broadcasting and virtual reality, yet its large data volume poses significant challenges for efficient storage and transmission. As deep contextu…