MOLTR: Multiple Object Localisation, Tracking, and Reconstruction from Monocular RGB Videos
Semantic aware reconstruction is more advantageous than geometric-only reconstruction for future robotic and AR/VR applications because it represents not only where things are, but also what things are. Object-centric mapping is a task to build an object-level reconstruction where objects are separate and meaningful entities that convey both geometry and semantic information. In this paper, we present MOLTR, a solution to object-centric mapping using only monocular image sequences and camera poses. It is able to localise, track, and reconstruct multiple objects in an online fashion when an RGB camera captures a video of the surrounding. Given a new RGB frame, MOLTR firstly applies a monocular 3D detector to localise objects of interest and extract their shape codes that represent the object shapes in a learned embedding space. Detections are then merged to existing objects in the map after data association. Motion state (i.e. kinematics and the motion status) of each object is tracked by a multiple model Bayesian filter and object shape is progressively refined by fusing multiple shape code. We evaluate localisation, tracking, and reconstruction on benchmarking datasets for indoor and outdoor scenes, and show superior performance over previous approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingObjectObject LocalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SLAM++: Simultaneous Localisation and Mapping at the Level of Objects
We present the major advantages of a new 'object oriented' 3D SLAM paradigm, which takes full advantage in the loop of prior knowledge that many scenes consist of repeated, domain-specific objects and structures. As a ha…
3D Object RecognitionDescriptiveObjectObject Recognition+1Tracking by Prediction: A Deep Generative Model for Mutli-Person localisation and Tracking
Current multi-person localisation and tracking systems have an over reliance on the use of appearance models for target re-identification and almost no approaches employ a complete deep learning solution for both objecti…
Generative Adversarial NetworkPedestrian Trajectory PredictionPerson Re-IdentificationTrajectory PredictionPerception Test 2025: Challenge Summary and a Unified VQA Extension
The Third Perception Test challenge was organised as a full-day workshop alongside the IEEE/CVF International Conference on Computer Vision (ICCV) 2025. Its primary goal is to benchmark state-of-the-art video models and …
Video GenerationObject TrackingPoint TrackingDevelopment and evaluation of automated localisation and reconstruction of all fruits on tomato plants in a greenhouse based on multi-view perception and 3D multi-object tracking
The ability to accurately represent and localise relevant objects is essential for robots to carry out tasks effectively. Traditional approaches, where robots simply capture an image, process that image to take an action…
3D Multi-Object TrackingAllMulti-Object TrackingObject TrackingPerception Test 2023: A Summary of the First Challenge And Outcome
The First Perception Test challenge was held as a half-day workshop alongside the IEEE/CVF International Conference on Computer Vision (ICCV) 2023, with the goal of benchmarking state-of-the-art video models on the recen…
BenchmarkingGrounded Video Question AnsweringMultiple-choiceObject Tracking+3