paper-with-me

Papers

Structured Semidefinite Programming for Recovering Structured Preconditioners

2023-10-27 · NeurIPS 2023 11

We develop a general framework for finding approximately-optimal preconditioners for solving linear systems. Leveraging this framework we obtain improved runtimes for fundamental preconditioning and linear system solving problems including the following. We give an algorithm which, given positive definite $\mathbf{K} \in \mathbb{R}^{d \times d}$ with $\mathrm{nnz}(\mathbf{K})$ nonzero entries, computes an $\epsilon$-optimal diagonal preconditioner in time $\widetilde{O}(\mathrm{nnz}(\mathbf{K}) \cdot \mathrm{poly}(\kappa^\star,\epsilon^{-1}))$, where $\kappa^\star$ is the optimal condition number of the rescaled matrix. We give an algorithm which, given $\mathbf{M} \in \mathbb{R}^{d \times d}$ that is either the pseudoinverse of a graph Laplacian matrix or a constant spectral approximation of one, solves linear systems in $\mathbf{M}$ in $\widetilde{O}(d^2)$ time. Our diagonal preconditioning results improve state-of-the-art runtimes of $\Omega(d^{3.5})$ attained by general-purpose semidefinite programming, and our solvers improve state-of-the-art runtimes of $\Omega(d^{\omega})$ where $\omega > 2.3$ is the current matrix multiplication constant. We attain our results via new algorithms for a class of semidefinite programs (SDPs) we call matrix-dictionary approximation SDPs, which we leverage to solve an associated problem we call matrix-dictionary recovery.

📄 PDF Abstract BibTeX arXiv:2310.18265

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Structured Preconditioners in Adaptive Optimization: A Unified Analysis

2025-03-13 · Shuo Xie, Tianhao Wang, Sashank Reddi, Sanjiv Kumar 외

We present a novel unified analysis for a broad class of adaptive optimization algorithms with structured (e.g., layerwise, diagonal, and kronecker-factored) preconditioners for both online regret minimization and offlin…

Precise Semidefinite Programming Formulation of Atomic Norm Minimization for Recovering d-Dimensional ($d\geq 2$) Off-the-Grid Frequencies

2013-12-02 · Weiyu Xu, Jian-Feng Cai, Kumar Vijay Mishra, Myung Cho 외

Recent research in off-the-grid compressed sensing (CS) has demonstrated that, under certain conditions, one can successfully recover a spectrally sparse signal from a few time-domain samples even though the dictionary i…

compressed sensing

The Mixing method: low-rank coordinate descent for semidefinite programming with diagonal constraints

2017-06-01 · Po-Wei Wang, Wei-Cheng Chang, J. Zico Kolter

In this paper, we propose a low-rank coordinate descent approach to structured semidefinite programming with diagonal constraints. The approach, which we call the Mixing method, is extremely simple to implement, has no f…

Learning Word EmbeddingsWord Embeddings

Achieving Exact Cluster Recovery Threshold via Semidefinite Programming

2014-11-24 · Bruce Hajek, Yihong Wu, Jiaming Xu

The binary symmetric stochastic block model deals with a random graph of $n$ vertices partitioned into two equal-sized clusters, such that each pair of vertices is connected independently with probability $p$ within clus…

Stochastic Block Model

Conic Descent Redux for Memory-Efficient Optimization

2023-08-13 · Bingcong Li, Georgios B. Giannakis

Conic programming has well-documented merits in a gamut of signal processing and machine learning tasks. This contribution revisits a recently developed first-order conic descent (CD) solver, and advances it in three asp…