IFFNeRF: Initialisation Free and Fast 6DoF pose estimation from a single image and a NeRF model
We introduce IFFNeRF to estimate the six degrees-of-freedom (6DoF) camera pose of a given image, building on the Neural Radiance Fields (NeRF) formulation. IFFNeRF is specifically designed to operate in real-time and eliminates the need for an initial pose guess that is proximate to the sought solution. IFFNeRF utilizes the Metropolis-Hasting algorithm to sample surface points from within the NeRF model. From these sampled points, we cast rays and deduce the color for each ray through pixel-level view synthesis. The camera pose can then be estimated as the solution to a Least Squares problem by selecting correspondences between the query image and the resulting bundle. We facilitate this process through a learned attention mechanism, bridging the query image embedding with the embedding of parameterized rays, thereby matching rays pertinent to the image. Through synthetic and real evaluation settings, we show that our method can improve the angular and translation error accuracy by 80.1% and 67.3%, respectively, compared to iNeRF while performing at 34fps on consumer hardware and not requiring the initial pose guess.
Code (0)
등록된 구현이 없습니다.
Tasks
NeRFPose EstimationSimilar Papers 제목 키워드 기반
Initialization Approach for Nonlinear State-Space Identification via the Subspace Encoder Approach
The SUBNET neural network architecture has been developed to identify nonlinear state-space models from input-output data. To achieve this, it combines the rolled-out nonlinear state-space equations and a state encoder f…
State Space ModelsHybrid One-Shot 3D Hand Pose Estimation by Exploiting Uncertainties
Model-based approaches to 3D hand tracking have been shown to perform well in a wide range of scenarios. However, they require initialisation and cannot recover easily from tracking failures that occur due to fast hand m…
3D Hand Pose EstimationHand Pose EstimationPose EstimationvalidOptimistic Exploration even with a Pessimistic Initialisation
Optimistic initialisation is an effective strategy for efficient exploration in reinforcement learning (RL). In the tabular case, all provably efficient model-free algorithms rely on it. However, model-free deep RL algor…
Efficient ExplorationQ-LearningReinforcement LearningReinforcement Learning (RL)A new initialisation to Control Gradients in Sinusoidal Neural network
Proper initialisation strategy is of primary importance to mitigate gradient explosion or vanishing when training neural networks. Yet, the impact of initialisation parameters still lacks a precise theoretical understand…
Image ReconstructionDiscoMatch: Fast Discrete Optimisation for Geometrically Consistent 3D Shape Matching
In this work we propose to combine the advantages of learningbased and combinatorial formalisms for 3D shape matching. While learningbased methods lead to state-of-the-art matching performance, they do not ensure geometr…
valid