PixTrack: Precise 6DoF Object Pose Tracking using NeRF Templates and Feature-metric Alignment
We present PixTrack, a vision based object pose tracking framework using novel view synthesis and deep feature-metric alignment. We follow an SfM-based relocalization paradigm where we use a Neural Radiance Field to canonically represent the tracked object. Our evaluations demonstrate that our method produces highly accurate, robust, and jitter-free 6DoF pose estimates of objects in both monocular RGB images and RGB-D images without the need of any data annotation or trajectory smoothing. Our method is also computationally efficient making it easy to have multi-object tracking with no alteration to our algorithm through simple CPU multiprocessing. Our code is available at: https://github.com/GiantAI/pixtrack
Code (1)
Tasks
CPUMulti-Object TrackingNeRFNovel View SynthesisObjectObject TrackingPose TrackingSimilar Papers 제목 키워드 기반
Open-NeRF: Towards Open Vocabulary NeRF Decomposition
In this paper, we address the challenge of decomposing Neural Radiance Fields (NeRF) into objects from an open vocabulary, a critical task for object manipulation in 3D reconstruction and view synthesis. Current techniqu…
3D ReconstructionNeRFSegmentationGear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling
Extensions of Neural Radiance Fields (NeRFs) to model dynamic scenes have enabled their near photo-realistic, free-viewpoint rendering. Although these methods have shown some potential in creating immersive experiences, …
Image SegmentationNeRFNovel View SynthesisSemantic SegmentationSLAIM: Robust Dense Neural SLAM for Online Tracking and Mapping
We present SLAIM - Simultaneous Localization and Implicit Mapping. We propose a novel coarse-to-fine tracking model tailored for Neural Radiance Field SLAM (NeRF-SLAM) to achieve state-of-the-art tracking performance. No…
NeRFExact-NeRF: An Exploration of a Precise Volumetric Parameterization for Neural Radiance Fields
Neural Radiance Fields (NeRF) have attracted significant attention due to their ability to synthesize novel scene views with great accuracy. However, inherent to their underlying formulation, the sampling of points along…
NeRFSurg-InvNeRF: Invertible NeRF for 3D tracking and reconstruction in surgical vision
We proposed a novel test-time optimisation (TTO) approach framed by a NeRF-based architecture for long-term 3D point tracking. Most current methods in point tracking struggle to obtain consistent motion or are limited to…
Point Tracking