Self-Supervised Scene Flow Estimation with Point-Voxel Fusion and Surface Representation
Scene flow estimation aims to generate the 3D motion field of points between two consecutive frames of point clouds, which has wide applications in various fields. Existing point-based methods ignore the irregularity of point clouds and have difficulty capturing long-range dependencies due to the inefficiency of point-level computation. Voxel-based methods suffer from the loss of detail information. In this paper, we propose a point-voxel fusion method, where we utilize a voxel branch based on sparse grid attention and the shifted window strategy to capture long-range dependencies and a point branch to capture fine-grained features to compensate for the information loss in the voxel branch. In addition, since xyz coordinates are difficult to describe the geometric structure of complex 3D objects in the scene, we explicitly encode the local surface information of the point cloud through the umbrella surface feature extraction (USFE) module. We verify the effectiveness of our method by conducting experiments on the Flyingthings3D and KITTI datasets. Our method outperforms all other self-supervised methods and achieves highly competitive results compared to fully supervised methods. We achieve improvements in all metrics, especially EPE, which is reduced by 8.51% on the KITTIo dataset and 10.52% on the KITTIs dataset, respectively.
Code (0)
등록된 구현이 없습니다.
Tasks
Scene Flow EstimationSelf-supervised Scene Flow EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Adversarial Self-Supervised Scene Flow Estimation
This work proposes a metric learning approach for self-supervised scene flow estimation. Scene flow estimation is the task of estimating 3D flow vectors for consecutive 3D point clouds. Such flow vectors are fruitful, \e…
Metric LearningScene Flow EstimationSelf-supervised Scene Flow EstimationTripletDo not trust the neighbors! Adversarial Metric Learning for Self-Supervised Scene Flow Estimation
Scene flow is the task of estimating 3D motion vectors to individual points of a dynamic 3D scene. Motion vectors have shown to be beneficial for downstream tasks such as action classification and collision avoidance. Ho…
Action ClassificationCollision AvoidanceMetric LearningScene Flow Estimation+2SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving
Scene flow estimation predicts the 3D motion at each point in successive LiDAR scans. This detailed, point-level, information can help autonomous vehicles to accurately predict and understand dynamic changes in their sur…
Autonomous DrivingAutonomous VehiclesObjectScene Flow Estimation+1RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior
In this work, we focus on scene flow learning on point clouds in a self-supervised manner. A real-world scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented a…
Motion EstimationSelf-Supervised LearningSelf-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior
In this article, we investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. A realistic scene can be well modeled as a collection of rigidly moving parts, therefore its …
Motion Estimationmotion predictionScene Flow EstimationSelf-Supervised Learning