MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration
With the recent advances in autonomous driving and the decreasing cost of LiDARs, the use of multimodal sensor systems is on the rise. However, in order to make use of the information provided by a variety of complimentary sensors, it is necessary to accurately calibrate them. We take advantage of recent advances in computer graphics and implicit volumetric scene representation to tackle the problem of multi-sensor spatial and temporal calibration. Thanks to a new formulation of the Neural Radiance Field (NeRF) optimization, we are able to jointly optimize calibration parameters along with scene representation based on radiometric and geometric measurements. Our method enables accurate and robust calibration from data captured in uncontrolled and unstructured urban environments, making our solution more scalable than existing calibration solutions. We demonstrate the accuracy and robustness of our method in urban scenes typically encountered in autonomous driving scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingModel OptimizationNeRFSimilar Papers 제목 키워드 기반
Dynamic CT Reconstruction from Limited Views with Implicit Neural Representations and Parametric Motion Fields
Reconstructing dynamic, time-varying scenes with computed tomography (4D-CT) is a challenging and ill-posed problem common to industrial and medical settings. Existing 4D-CT reconstructions are designed for sparse sampli…
CT Reconstruction3DGS-Calib: 3D Gaussian Splatting for Multimodal SpatioTemporal Calibration
Reliable multimodal sensor fusion algorithms require accurate spatiotemporal calibration. Recently, targetless calibration techniques based on implicit neural representations have proven to provide precise and robust res…
3DGSSensor FusionS-INF: Towards Realistic Indoor Scene Synthesis via Scene Implicit Neural Field
Learning-based methods have become increasingly popular in 3D indoor scene synthesis (ISS), showing superior performance over traditional optimization-based approaches. These learning-based methods typically model distri…
Indoor Scene SynthesisObjectSce2DriveX: A Generalized MLLM Framework for Scene-to-Drive Learning
End-to-end autonomous driving, which directly maps raw sensor inputs to low-level vehicle controls, is an important part of Embodied AI. Despite successes in applying Multimodal Large Language Models (MLLMs) for high-lev…
Autonomous DrivingBench2DriveMotion PlanningQuestion Answering+3LLaVA-4D: Embedding SpatioTemporal Prompt into LMMs for 4D Scene Understanding
Despite achieving significant progress in 2D image understanding, large multimodal models (LMMs) struggle in the physical world due to the lack of spatial representation. Typically, existing 3D LMMs mainly embed 3D posit…
Scene Understanding