Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks
Exploiting past 3D LiDAR scans to predict future point clouds is a promising method for autonomous mobile systems to realize foresighted state estimation, collision avoidance, and planning. In this paper, we address the problem of predicting future 3D LiDAR point clouds given a sequence of past LiDAR scans. Estimating the future scene on the sensor level does not require any preceding steps as in localization or tracking systems and can be trained self-supervised. We propose an end-to-end approach that exploits a 2D range image representation of each 3D LiDAR scan and concatenates a sequence of range images to obtain a 3D tensor. Based on such tensors, we develop an encoder-decoder architecture using 3D convolutions to jointly aggregate spatial and temporal information of the scene and to predict the future 3D point clouds. We evaluate our method on multiple datasets and the experimental results suggest that our method outperforms existing point cloud prediction architectures and generalizes well to new, unseen environments without additional fine-tuning. Our method operates online and is faster than the common LiDAR frame rate of 10 Hz.
Code (1)
Tasks
Collision AvoidanceDecoderState EstimationSimilar Papers 제목 키워드 기반
Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos
Recently, the community has made tremendous progress in developing effective methods for point cloud video understanding that learn from massive amounts of labeled data. However, annotating point cloud videos is usually …
point cloud video understandingSelf-Supervised LearningVideo UnderstandingContrastive Predictive Autoencoders for Dynamic Point Cloud Self-Supervised Learning
We present a new self-supervised paradigm on point cloud sequence understanding. Inspired by the discriminative and generative self-supervised methods, we design two tasks, namely point cloud sequence based Contrastive P…
Action RecognitionColorizationContrastive LearningDecoder+2PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videos
Self-supervised learning can extract representations of good quality from solely unlabeled data, which is appealing for point cloud videos due to their high labelling cost. In this paper, we propose a contrastive mask pr…
Self-Supervised LearningTransfer LearningSpatio-temporal Self-Supervised Representation Learning for 3D Point Clouds
To date, various 3D scene understanding tasks still lack practical and generalizable pre-trained models, primarily due to the intricate nature of 3D scene understanding tasks and their immense variations introduced by ca…
3D Object Detection3D Point Cloud Classification3D Point Cloud Linear Classification3D Semantic Segmentation+8Self-Supervised Learning of Part Mobility from Point Cloud Sequence
Part mobility analysis is a significant aspect required to achieve a functional understanding of 3D objects. It would be natural to obtain part mobility from the continuous part motion of 3D objects. In this study, we in…
Self-Supervised Learning