Residual Expansion Algorithm: Fast and Effective Optimization for Nonconvex Least Squares Problems
We propose the residual expansion (RE) algorithm: a global (or near-global) optimization method for nonconvex least squares problems. Unlike most existing nonconvex optimization techniques, the RE algorithm is not based on either stochastic or multi-point searches; therefore, it can achieve fast global optimization. Moreover, the RE algorithm is easy to implement and successful in high-dimensional optimization. The RE algorithm exhibits excellent empirical performance in terms of k-means clustering, point-set registration, optimized product quantization, and blind image deblurring.
Code (0)
등록된 구현이 없습니다.
Tasks
Blind Image DeblurringClusteringDeblurringglobal-optimizationImage DeblurringQuantizationSimilar Papers 제목 키워드 기반
Taming Gradient Oversmoothing and Expansion in Graph Neural Networks
Oversmoothing has been claimed as a primary bottleneck for multi-layered graph neural networks (GNNs). Multiple analyses have examined how and why oversmoothing occurs. However, none of the prior work addressed how optim…
Deep Sparse Coding Using Optimized Linear Expansion of Thresholds
We address the problem of reconstructing sparse signals from noisy and compressive measurements using a feed-forward deep neural network (DNN) with an architecture motivated by the iterative shrinkage-thresholding algori…
DenoisingImage DenoisingA Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networks
The scaling behavior, in which test performance often improves as model size and data increase, is a central empirical phenomenon in modern deep learning, yet its theoretical basis remains incomplete. In this paper, we s…
Online Optimization with Costly and Noisy Measurements using Random Fourier Expansions
This paper analyzes DONE, an online optimization algorithm that iteratively minimizes an unknown function based on costly and noisy measurements. The algorithm maintains a surrogate of the unknown function in the form of…
Bayesian OptimizationWhen Good Enough Is Optimal: Multiplication-Only Matrix Inversion Approximation for Quantized Gated DeltaNet
Matrix inversion in chunk-wise parallel linear attention is a major bottleneck for long-context modeling, particularly on NPUs, where forward-substitution-based methods exhibit limited parallelism and poor hardware utili…