Learning Spatial-Temporal Embeddings for Sequential Point Cloud Frame Interpolation
A point cloud sequence is usually acquired at a low frame rate owing to the limitations from the sensing equipment. Consequently, the immersive experience of the virtual reality might be greatly degraded. To tackle this issue, a point cloud frame interpolation process can be used to increase the frame rate of the acquired point cloud sequence by generating new frames between the consecutive ones. However, it is still challenging for deep neural networks to synthesize high-fidelity point clouds, especially for those with complex geometric details and large motion. In this paper, a novel frame interpolation network is proposed, which jointly exploits the spatial features and flows. The key success of our method lies in the developed spatial-temporal feature propagation module and temporal-aware feature-to-point mapping module. The former effectively embeds the spatial features and scene flows into a spatial-temporal feature representation (STFR). The latter generates a much improved target frame from STFR. Extensive experimental results have demonstrated that our method has achieved the best performance in most cases.
Code (1)
Tasks
3D Point Cloud InterpolationSimilar Papers 제목 키워드 기반
Paving the Way for Point Cloud Video Representation Learning Using A PDE Model
Investigating spatial-temporal correlations, specifically how spatial points vary over time, is crucial for understanding point cloud videos. Traditional methods, particularly flow-based techniques, struggle with these c…
Representation LearningContrastive LearningSequential 3D Human Pose and Shape Estimation From Point Clouds
This work addresses the problem of 3D human pose and shape estimation from a sequence of point clouds. Existing sequential 3D human shape estimation methods mainly focus on the template model fitting from a sequence of d…
3D human pose and shape estimation3D Human Shape EstimationUST-SSM: Unified Spatio-Temporal State Space Models for Point Cloud Video Modeling
Point cloud videos capture dynamic 3D motion while reducing the effects of lighting and viewpoint variations, making them highly effective for recognizing subtle and continuous human actions. Although Selective State Spa…
DSLO: Deep Sequence LiDAR Odometry Based on Inconsistent Spatio-temporal Propagation
This paper introduces a 3D point cloud sequence learning model based on inconsistent spatio-temporal propagation for LiDAR odometry, termed DSLO. It consists of a pyramid structure with a spatial information reuse strate…
RTEPose-Graph Attentional Graph Neural Network for Lidar Place Recognition
This paper proposes a pose-graph attentional graph neural network, called P-GAT, which compares (key)nodes between sequential and non-sequential sub-graphs for place recognition tasks as opposed to a common frame-to-fram…
Domain AdaptationGraph Neural NetworkRetrieval