paper-with-me

Papers

Variable-Wise Diagonal Preconditioning for Primal-Dual Splitting: Design and Applications

2023-01-20 · Kazuki Naganuma, Shunsuke Ono

This paper proposes a method for designing diagonal preconditioners for a preconditioned primal-dual splitting method (P-PDS), an efficient algorithm that solves nonsmooth convex optimization problems. To speed up the convergence of P-PDS, a design method has been proposed to automatically determine appropriate preconditioners from the problem structure. However, the existing method has two limitations. One is that it directly accesses all elements of matrices representing linear operators involved in a given problem, which is inconvenient for handling linear operators implemented as procedures rather than matrices. The other is that it takes an element-wise preconditioning approach, which turns certain types of proximity operators into analytically intractable forms. To overcome these limitations, we establish an Operator norm-based design method of Variable-wise Diagonal Preconditioning (OVDP). First, OVDP constructs diagonal preconditioners using only (upper bounds) of the operator norms of linear operators, thus eliminating the need for their explicit matrix representations. Furthermore, since OVDP takes a variable-wise preconditioning approach, it keeps any proximity operator analytically computable. We also prove that our preconditioners satisfy the convergence condition of P-PDS. Finally, we demonstrate the effectiveness and usefulness of OVDP through applications to mixed noise removal of hyperspectral images, hyperspectral unmixing, and graph signal recovery.

📄 PDF Abstract BibTeX arXiv:2301.08468

Code (0)

등록된 구현이 없습니다.

Tasks

Hyperspectral Unmixing

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Similar Papers 제목 키워드 기반

Optimal Diagonal Preconditioning

2022-09-02 · Zhaonan Qu, Wenzhi Gao, Oliver Hinder, Yinyu Ye 외

Preconditioning has long been a staple technique in optimization, often applied to reduce the condition number of a matrix and speed up the convergence of algorithms. Although there are many popular preconditioning techn…

On The Concurrence of Layer-wise Preconditioning Methods and Provable Feature Learning

2025-02-03 · Thomas T. Zhang, Behrad Moniri, Ansh Nagwekar, Faraz Rahman 외

Layer-wise preconditioning methods are a family of memory-efficient optimization algorithms that introduce preconditioners per axis of each layer's weight tensors. These methods have seen a recent resurgence, demonstrati…

Representation Learning

Optimal Diagonal Preconditioning Beyond Worst-Case Conditioning: Theory and Practice of Omega Scaling

2025-09-27 · Saeed Ghadimi, Woosuk L. Jung, Arnesh Sujanani, David Torregrosa-Belén 외 arxiv

We study optimal diagonal preconditioning using the classical worst-case $κ$-condition number and the averaging-based $ω$-condition number. For the $κ$-optimal preconditioning problem, we derive an affine-based pseudocon…

Matrix-Free Preconditioning in Online Learning

2019-05-29 · Ashok Cutkosky, Tamas Sarlos

We provide an online convex optimization algorithm with regret that interpolates between the regret of an algorithm using an optimal preconditioning matrix and one using a diagonal preconditioning matrix. Our regret boun…

Benchmarking

High-dimensional Adaptive MCMC with Reduced Computational Complexity

2026-04-10 · Max Hird, Samuel Livingstone arxiv

We propose an adaptive MCMC method that learns a linear preconditioner which is dense in its off-diagonal elements but sparse in its parametrisation. Due to this sparsity, we achieve a per-iteration computational complex…