Pixel-Perfect Structure-from-Motion with Featuremetric Refinement
Finding local features that are repeatable across multiple views is a cornerstone of sparse 3D reconstruction. The classical image matching paradigm detects keypoints per-image once and for all, which can yield poorly-localized features and propagate large errors to the final geometry. In this paper, we refine two key steps of structure-from-motion by a direct alignment of low-level image information from multiple views: we first adjust the initial keypoint locations prior to any geometric estimation, and subsequently refine points and camera poses as a post-processing. This refinement is robust to large detection noise and appearance changes, as it optimizes a featuremetric error based on dense features predicted by a neural network. This significantly improves the accuracy of camera poses and scene geometry for a wide range of keypoint detectors, challenging viewing conditions, and off-the-shelf deep features. Our system easily scales to large image collections, enabling pixel-perfect crowd-sourced localization at scale. Our code is publicly available at https://github.com/cvg/pixel-perfect-sfm as an add-on to the popular SfM software COLMAP.
Code (3)
Tasks
3D ReconstructionSimilar Papers 제목 키워드 기반
Uncertainty-Driven Dense Two-View Structure from Motion
This work introduces an effective and practical solution to the dense two-view structure from motion (SfM) problem. One vital question addressed is how to mindfully use per-pixel optical flow correspondence between two f…
Depth EstimationOptical Flow EstimationPose EstimationVocal Bursts Valence PredictionPixPerfect: Seamless Latent Diffusion Local Editing with Discriminative Pixel-Space Refinement
Latent Diffusion Models (LDMs) have markedly advanced the quality of image inpainting and local editing. However, the inherent latent compression often introduces pixel-level inconsistencies, such as chromatic shifts, te…
Image InpaintingImage EditingFeaturemetric benchmarking: Quantum computer benchmarks based on circuit features
Benchmarks that concisely summarize the performance of many-qubit quantum computers are essential for measuring progress towards the goal of useful quantum computation. In this work, we present a benchmarking framework t…
BenchmarkingPerfecting Depth: Uncertainty-Aware Enhancement of Metric Depth
We propose a novel two-stage framework for sensor depth enhancement, called Perfecting Depth. This framework leverages the stochastic nature of diffusion models to automatically detect unreliable depth regions while pres…
Autonomous DrivingRefineAnything: Multimodal Region-Specific Refinement for Perfect Local Details
We introduce region-specific image refinement as a dedicated problem setting: given an input image and a user-specified region (e.g., a scribble mask or a bounding box), the goal is to restore fine-grained details while …
Image Generation