PerMO: Perceiving More at Once from a Single Image for Autonomous Driving
We present a novel approach to detect, segment, and reconstruct complete textured 3D models of vehicles from a single image for autonomous driving. Our approach combines the strengths of deep learning and the elegance of traditional techniques from part-based deformable model representation to produce high-quality 3D models in the presence of severe occlusions. We present a new part-based deformable vehicle model that is used for instance segmentation and automatically generate a dataset that contains dense correspondences between 2D images and 3D models. We also present a novel end-to-end deep neural network to predict dense 2D/3D mapping and highlight its benefits. Based on the dense mapping, we are able to compute precise 6-DoF poses and 3D reconstruction results at almost interactive rates on a commodity GPU. We have integrated these algorithms with an autonomous driving system. In practice, our method outperforms the state-of-the-art methods for all major vehicle parsing tasks: 2D instance segmentation by 4.4 points (mAP), 6-DoF pose estimation by 9.11 points, and 3D detection by 1.37. Moreover, we have released all of the source code, dataset, and the trained model on Github.
Code (0)
등록된 구현이 없습니다.
Tasks
3D ReconstructionAutonomous DrivingGPUInstance SegmentationPose EstimationSemantic SegmentationSimilar Papers 제목 키워드 기반
Approximate Supermodularity Bounds for Experimental Design
This work provides performance guarantees for the greedy solution of experimental design problems. In particular, it focuses on A- and E-optimal designs, for which typical guarantees do not apply since the mean-square er…
Experimental DesignSlack and Margin Rescaling as Convex Extensions of Supermodular Functions
Slack and margin rescaling are variants of the structured output SVM, which is frequently applied to problems in computer vision such as image segmentation, object localization, and learning parts based object models. Th…
Image SegmentationObject LocalizationSemantic SegmentationStructured PredictionExistence and structure of Nash equilibria for supermodular games
Two theorems announced by Topkis about the topological description of sublattices are proved. They are applied to extend some classical results concerning the existence and the order structure of Nash equilibria of certa…
Learning the Effect of Registration Hyperparameters with HyperMorph
We introduce HyperMorph, a framework that facilitates efficient hyperparameter tuning in learning-based deformable image registration. Classical registration algorithms perform an iterative pair-wise optimization to comp…
Image RegistrationHyperMotion: DiT-Based Pose-Guided Human Image Animation of Complex Motions
Recent advances in diffusion models have significantly improved conditional video generation, particularly in the pose-guided human image animation task. Although existing methods are capable of generating high-fidelity …
Image AnimationVideo Generation