SMORE: Simultaneous Map and Object REconstruction
We present a method for dynamic surface reconstruction of large-scale urban scenes from LiDAR. Depth-based reconstructions tend to focus on small-scale objects or large-scale SLAM reconstructions that treat moving objects as outliers. We take a holistic perspective and optimize a compositional model of a dynamic scene that decomposes the world into rigidly-moving objects and the background. To achieve this, we take inspiration from recent novel view synthesis methods and frame the reconstruction problem as a global optimization over neural surfaces, ego poses, and object poses, which minimizes the error between composed spacetime surfaces and input LiDAR scans. In contrast to view synthesis methods, which typically minimize 2D errors with gradient descent, we minimize a 3D point-to-surface error by coordinate descent, which we decompose into registration and surface reconstruction steps. Each step can be handled well by off-the-shelf methods without any re-training. We analyze the surface reconstruction step for rolling-shutter LiDARs, and show that deskewing operations common in continuous time SLAM can be applied to dynamic objects as well, improving results over prior art by an order of magnitude. Beyond pursuing dynamic reconstruction as a goal in and of itself, we propose that such a system can be used to auto-label partially annotated sequences and produce ground truth annotation for hard-to-label problems such as depth completion and scene flow. Please see https://anishmadan23.github.io/smore/ for more visual results.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth CompletionDynamic Reconstructionglobal-optimizationMotion CompensationNovel View SynthesisObjectObject ReconstructionSurface ReconstructionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learning
Offline Goal-Conditioned Reinforcement Learning (GCRL) is tasked with learning to achieve multiple goals in an environment purely from offline datasets using sparse reward functions. Offline GCRL is pivotal for developin…
Contrastive Learningreinforcement-learningReinforcement LearningRobot ManipulationSpectrum-based Modality Representation Fusion Graph Convolutional Network for Multimodal Recommendation
Incorporating multi-modal features as side information has recently become a trend in recommender systems. To elucidate user-item preferences, recent studies focus on fusing modalities via concatenation, element-wise sum…
Graph LearningMultimodal RecommendationRecommendation SystemsLess Is More: Fast and Accurate Reasoning with Cross-Head Unified Sparse Attention
Large reasoning models achieve strong performance through test-time scaling, but this incurs substantial computational overhead due to long decoding from short prompts. While sparse attention can reduce latency and memor…
SMORE: Knowledge Graph Completion and Multi-hop Reasoning in Massive Knowledge Graphs
Knowledge graphs (KGs) capture knowledge in the form of head--relation--tail triples and are a crucial component in many AI systems. There are two important reasoning tasks on KGs: (1) single-hop knowledge graph completi…
CPUGPUKnowledge Graph CompletionKnowledge Graphs+1SMORE: Similarity-based Hyperdimensional Domain Adaptation for Multi-Sensor Time Series Classification
Many real-world applications of the Internet of Things (IoT) employ machine learning (ML) algorithms to analyze time series information collected by interconnected sensors. However, distribution shift, a fundamental chal…
Domain AdaptationTime SeriesTime Series Classification