paper-with-me

홈 › Papers

Low-Rank Updates of Matrix Square Roots

2022-01-31 · Shany Shumeli, Petros Drineas, Haim Avron

Models in which the covariance matrix has the structure of a sparse matrix plus a low rank perturbation are ubiquitous in data science applications. It is often desirable for algorithms to take advantage of such structures, avoiding costly matrix computations that often require cubic time and quadratic storage. This is often accomplished by performing operations that maintain such structures, e.g. matrix inversion via the Sherman-Morrison-Woodbury formula. In this paper we consider the matrix square root and inverse square root operations. Given a low rank perturbation to a matrix, we argue that a low-rank approximate correction to the (inverse) square root exists. We do so by establishing a geometric decay bound on the true correction's eigenvalues. We then proceed to frame the correction as the solution of an algebraic Riccati equation, and discuss how a low-rank solution to that equation can be computed. We analyze the approximation error incurred when approximately solving the algebraic Riccati equation, providing spectral and Frobenius norm forward and backward error bounds. Finally, we describe several applications of our algorithms, and demonstrate their utility in numerical experiments.

📄 PDF Abstract BibTeX arXiv:2201.13156

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Vector logic allows counterfactual virtualization by The Square Root of NOT

2020-03-09 · Eduardo Mizraji

In this work we investigate the representation of counterfactual conditionals using the vector logic, a matrix-vectors formalism for logical functions and truth values. Inside this formalism, the counterfactuals can be t…

counterfactual

PRISM: Distribution-free Adaptive Computation of Matrix Functions for Accelerating Neural Network Training

2026-01-29 · Shenghao Yang, Zhichao Wang, Oleg Balabanov, N. Benjamin Erichson 외 arxiv

Matrix functions such as square root, inverse roots, and orthogonalization play a central role in preconditioned gradient methods for neural network training. This has motivated the development of iterative algorithms th…

Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization

2020-06-19 · NeurIPS 2020 12 · Geoff Pleiss, Martin Jankowiak, David Eriksson, Anil Damle 외

Matrix square roots and their inverses arise frequently in machine learning, e.g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance ma…

Bayesian OptimizationGaussian Processessubspace methods

Back to Square Roots: An Optimal Bound on the Matrix Factorization Error for Multi-Epoch Differentially Private SGD

2025-05-17 · Nikita P. Kalinin, Ryan McKenna, Jalaj Upadhyay, Christoph H. Lampert

Matrix factorization mechanisms for differentially private training have emerged as a promising approach to improve model utility under privacy constraints. In practical settings, models are typically trained over multip…

Sparse Factorization of Large Square Matrices

2021-09-16 · Ruslan Khalitov, Tong Yu, Lei Cheng, Zhirong Yang

Square matrices appear in many machine learning problems and models. Optimization over a large square matrix is expensive in memory and in time. Therefore an economic approximation is needed. Conventional approximation a…

Long-range modeling