paper-with-me

홈 › Papers

Progressive Batching for Efficient Non-linear Least Squares

2020-10-21 · Huu Le, Christopher Zach, Edward Rosten, Oliver J. Woodford

Non-linear least squares solvers are used across a broad range of offline and real-time model fitting problems. Most improvements of the basic Gauss-Newton algorithm tackle convergence guarantees or leverage the sparsity of the underlying problem structure for computational speedup. With the success of deep learning methods leveraging large datasets, stochastic optimization methods received recently a lot of attention. Our work borrows ideas from both stochastic machine learning and statistics, and we present an approach for non-linear least-squares that guarantees convergence while at the same time significantly reduces the required amount of computation. Empirical results show that our proposed method achieves competitive convergence rates compared to traditional second-order approaches on common computer vision problems, such as image alignment and essential matrix estimation, with very large numbers of residuals.

📄 PDF Abstract BibTeX arXiv:2010.10968

Code (1)

intellhave/ProBLM 공식 구현

Tasks

Stochastic Optimization

Similar Papers 제목 키워드 기반

Parallelizing Stochastic Gradient Descent for Least Squares Regression: mini-batching, averaging, and model misspecification

2016-10-12 · Prateek Jain, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli 외

This work characterizes the benefits of averaging schemes widely used in conjunction with stochastic gradient descent (SGD). In particular, this work provides a sharp analysis of: (1) mini-batching, a method of averaging…

regression

Error dynamics of mini-batch gradient descent with random reshuffling for least squares regression

2024-06-06 · Jackie Lok, Rishi Sonthalia, Elizaveta Rebrova

We study the discrete dynamics of mini-batch gradient descent with random reshuffling for least squares regression. We show that the training and generalization errors depend on a sample cross-covariance matrix $Z$ betwe…

regression

Beating SGD Saturation with Tail-Averaging and Minibatching

2019-02-22 · NeurIPS 2019 12 · Nicole Mücke, Gergely Neu, Lorenzo Rosasco

While stochastic gradient descent (SGD) is one of the major workhorses in machine learning, the learning properties of many practically used variants are poorly understood. In this paper, we consider least squares learni…

An Even More Optimal Stochastic Optimization Algorithm: Minibatching and Interpolation Learning

2021-06-04 · NeurIPS 2021 12 · Blake Woodworth, Nathan Srebro

We present and analyze an algorithm for optimizing smooth and convex or strongly convex objectives using minibatch stochastic gradient estimates. The algorithm is optimal with respect to its dependence on both the miniba…

Stochastic Optimization

Online Reweighted Least Squares Algorithm for Sparse Recovery and Application to Short-Wave Infrared Imaging

2017-06-29 · Subhadip Mukherjee, Deepak R., Huaijin Chen, Ashok Veeraraghavan 외

We address the problem of sparse recovery in an online setting, where random linear measurements of a sparse signal are revealed sequentially and the objective is to recover the underlying signal. We propose a reweighted…