MambaFlow: A Novel and Flow-guided State Space Model for Scene Flow Estimation
Scene flow estimation aims to predict 3D motion from consecutive point cloud frames, which is of great interest in autonomous driving field. Existing methods face challenges such as insufficient spatio-temporal modeling and inherent loss of fine-grained feature during voxelization. However, the success of Mamba, a representative state space model (SSM) that enables global modeling with linear complexity, provides a promising solution. In this paper, we propose MambaFlow, a novel scene flow estimation network with a mamba-based decoder. It enables deep interaction and coupling of spatio-temporal features using a well-designed backbone. Innovatively, we steer the global attention modeling of voxel-based features with point offset information using an efficient Mamba-based decoder, learning voxel-to-point patterns that are used to devoxelize shared voxel representations into point-wise features. To further enhance the model's generalization capabilities across diverse scenarios, we propose a novel scene-adaptive loss function that automatically adapts to different motion patterns.Extensive experiments on the Argoverse 2 benchmark demonstrate that MambaFlow achieves state-of-the-art performance with real-time inference speed among existing works, enabling accurate flow estimation in real-world urban scenarios. The code is available at https://github.com/SCNU-RISLAB/MambaFlow.
Code (1)
Tasks
Autonomous DrivingDecoderMambaScene Flow EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MambaFlow: A Mamba-Centric Architecture for End-to-End Optical Flow Estimation
Recently, the Mamba architecture has demonstrated significant successes in various computer vision tasks, such as classification and segmentation. However, its application to optical flow estimation remains unexplored. I…
MambaOptical Flow EstimationOcclusion Guided Scene Flow Estimation on 3D Point Clouds
3D scene flow estimation is a vital tool in perceiving our environment given depth or range sensors. Unlike optical flow, the data is usually sparse and in most cases partially occluded in between two temporal samplings.…
Optical Flow EstimationScene Flow EstimationLearning Temporal 3D Semantic Scene Completion via Optical Flow Guidance
3D Semantic Scene Completion (SSC) provides comprehensive scene geometry and semantics for autonomous driving perception, which is crucial for enabling accurate and reliable decision-making. However, existing SSC methods…
3D Semantic Scene CompletionAutonomous DrivingOptical Flow EstimationOcclusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds
Understanding the flow in 3D space of sparsely sampled points between two consecutive time frames is the core stone of modern geometric-driven systems such as VR/AR, Robotics, and Autonomous driving. The lack of real, no…
Autonomous DrivingScene Flow EstimationSelf-supervised Scene Flow EstimationConsistency Guided Scene Flow Estimation
Consistency Guided Scene Flow Estimation (CGSF) is a self-supervised framework for the joint reconstruction of 3D scene structure and motion from stereo video. The model takes two temporal stereo pairs as input, and pred…
Scene Flow Estimation