paper-with-me

홈 › Papers

Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems

2020-04-15 · Zheng Dang, Kwang Moo Yi, Yinlin Hu, Fei Wang, Pascal Fua, Mathieu Salzmann

Many classical Computer Vision problems, such as essential matrix computation and pose estimation from 3D to 2D correspondences, can be tackled by solving a linear least-square problem, which can be done by finding the eigenvector corresponding to the smallest, or zero, eigenvalue of a matrix representing a linear system. Incorporating this in deep learning frameworks would allow us to explicitly encode known notions of geometry, instead of having the network implicitly learn them from data. However, performing eigendecomposition within a network requires the ability to differentiate this operation. While theoretically doable, this introduces numerical instability in the optimization process in practice. In this paper, we introduce an eigendecomposition-free approach to training a deep network whose loss depends on the eigenvector corresponding to a zero eigenvalue of a matrix predicted by the network. We demonstrate that our approach is much more robust than explicit differentiation of the eigendecomposition using two general tasks, outlier rejection and denoising, with several practical examples including wide-baseline stereo, the perspective-n-point problem, and ellipse fitting. Empirically, our method has better convergence properties and yields state-of-the-art results.

📄 PDF Abstract BibTeX arXiv:2004.07931

Code (0)

등록된 구현이 없습니다.

Tasks

DenoisingPose Estimation

Similar Papers 제목 키워드 기반

Whitening-Free Least-Squares Non-Gaussian Component Analysis

2016-03-03 · Hiroaki Shiino, Hiroaki Sasaki, Gang Niu, Masashi Sugiyama

Non-Gaussian component analysis (NGCA) is an unsupervised linear dimension reduction method that extracts low-dimensional non-Gaussian "signals" from high-dimensional data contaminated with Gaussian noise. NGCA can be re…

Dimensionality Reduction

PRISM: Distribution-free Adaptive Computation of Matrix Functions for Accelerating Neural Network Training

2026-01-29 · Shenghao Yang, Zhichao Wang, Oleg Balabanov, N. Benjamin Erichson 외 arxiv

Matrix functions such as square root, inverse roots, and orthogonalization play a central role in preconditioned gradient methods for neural network training. This has motivated the development of iterative algorithms th…

Doubly Accelerated Methods for Faster CCA and Generalized Eigendecomposition

2016-07-20 · ICML 2017 8 · Zeyuan Allen-Zhu, Yuanzhi Li

We study $k$-GenEV, the problem of finding the top $k$ generalized eigenvectors, and $k$-CCA, the problem of finding the top $k$ vectors in canonical-correlation analysis. We propose algorithms $\mathtt{LazyEV}$ and $\ma…

Eigendecomposition-free Training of Deep Networks with Zero Eigenvalue-based Losses

2018-03-21 · ECCV 2018 9 · Zheng Dang, Kwang Moo Yi, Yinlin Hu, Fei Wang 외

Many classical Computer Vision problems, such as essential matrix computation and pose estimation from 3D to 2D correspondences, can be solved by finding the eigenvector corresponding to the smallest, or zero, eigenvalue…

3D Pose EstimationPose Estimation

Scalable Derivative-Free Optimization for Nonlinear Least-Squares Problems

2020-07-26 · Coralia Cartis, Tyler Ferguson, Lindon Roberts

Derivative-free - or zeroth-order - optimization (DFO) has gained recent attention for its ability to solve problems in a variety of application areas, including machine learning, particularly involving objectives which …

Dimensionality Reduction