Non-smooth Variable Projection
Variable projection solves structured optimization problems by completely minimizing over a subset of the variables while iterating over the remaining variables. Over the last 30 years, the technique has been widely used, with empirical and theoretical results demonstrating both greater efficacy and greater stability compared to competing approaches. Classic examples have exploited closed-form projections and smoothness of the objective function. We extend the approach to problems that include non-smooth terms, and where the projection subproblems can only be solved inexactly by iterative methods. We propose an inexact adaptive algonrithm for solving such problems and analyze its computational complexity. Finally, we show how the theory can be used to design methods for selected problems occurring frequently in machine-learning and inverse problems.
Code (1)
Similar Papers 제목 키워드 기반
Continuous-Discrete Filtering and Smoothing on Submanifolds of Euclidean Space
In this paper the issue of filtering and smoothing in continuous discrete time is studied when the state variable evolves in some submanifold of Euclidean space, which may not have the usual Lebesgue measure. Formal expr…
Constrained Variable Projection for Structured Problems
Variable projection is a classical technique for separable nonlinear least-squares problems, in which variables that enter linearly are eliminated exactly, yielding a reduced nonlinear problem. By expressing this framewo…
Bilevel OptimizationFew-Shot LearningBoost Like a (Var)Pro: Trust-Region Gradient Boosting via Variable Projection
Gradient boosting, a method of building additive ensembles from weak learners, has established itself as a practical and theoretically-motivated approach to approximate functions, especially using decision tree weak lear…
Semismooth Newton Algorithm for Efficient Projections onto $\ell_{1, \infty}$-norm Ball
Structured sparsity-inducing $\ell_{1, \infty}$-norm, as a generalization of the classical $\ell_1$-norm, plays an important role in jointly sparse models which select or remove simultaneously all the variables forming a…
Faster PET Reconstruction with Non-Smooth Priors by Randomization and Preconditioning
Uncompressed clinical data from modern positron emission tomography (PET) scanners are very large, exceeding 350 million data points (projection bins). The last decades have seen tremendous advancements in mathematical i…