Expanded Alternating Optimization of Nonconvex Functions with Applications to Matrix Factorization and Penalized Regression
We propose a general technique for improving alternating optimization (AO) of nonconvex functions. Starting from the solution given by AO, we conduct another sequence of searches over subspaces that are both meaningful to the optimization problem at hand and different from those used by AO. To demonstrate the utility of our approach, we apply it to the matrix factorization (MF) algorithm for recommender systems and the coordinate descent algorithm for penalized regression (PR), and show meaningful improvements using both real-world (for MF) and simulated (for PR) data sets. Moreover, we demonstrate for MF that, by constructing search spaces customized to the given data set, we can significantly increase the convergence rate of our technique.
Code (0)
등록된 구현이 없습니다.
Tasks
Recommendation SystemsregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Global convergence of splitting methods for nonconvex composite optimization
We consider the problem of minimizing the sum of a smooth function $h$ with a bounded Hessian, and a nonsmooth function. We assume that the latter function is a composition of a proper closed function $P$ and a surjectiv…
Sparse Reduced Rank Regression With Nonconvex Regularization
In this paper, the estimation problem for sparse reduced rank regression (SRRR) model is considered. The SRRR model is widely used for dimension reduction and variable selection with applications in signal processing, ec…
Dimensionality ReductionEconometricsregressionVariable SelectionAccelerated Proximal Alternating Gradient-Descent-Ascent for Nonconvex Minimax Machine Learning
Alternating gradient-descent-ascent (AltGDA) is an optimization algorithm that has been widely used for model training in various machine learning applications, which aims to solve a nonconvex minimax optimization proble…
BIG-bench Machine LearningA Framework of Inertial Alternating Direction Method of Multipliers for Non-Convex Non-Smooth Optimization
In this paper, we propose an algorithmic framework, dubbed inertial alternating direction methods of multipliers (iADMM), for solving a class of nonconvex nonsmooth multiblock composite optimization problems with linear …
An Alternating Optimization Method for Bilevel Problems under the Polyak-Łojasiewicz Condition
Bilevel optimization has recently regained interest owing to its applications in emerging machine learning fields such as hyperparameter optimization, meta-learning, and reinforcement learning. Recent results have shown …