Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold

2018-11-02 · Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, Tong Zhang

We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of problems can be classified into three classes. Algorithms in the first class rely on information of the subgradients of the objective function and thus tend to converge slowly in practice. Algorithms in the second class are proximal point algorithms, which involve subproblems that can be as difficult as the original problem. Algorithms in the third class are based on operator-splitting techniques, but they usually lack rigorous convergence guarantees. In this paper, we propose a retraction-based proximal gradient method for solving this class of problems. We prove that the proposed method globally converges to a stationary point. Iteration complexity for obtaining an $\epsilon$-stationary solution is also analyzed. Numerical results on solving sparse PCA and compressed modes problems are reported to demonstrate the advantages of the proposed method.

📄 PDF Abstract BibTeX arXiv:1811.00980

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

PCA Principle Components Analysis (PCA) is an unsupervised method primary used for dimensionality reduction within machine learning. PCA is calculated via a singular value…

Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold

2020-05-03 · Bokun Wang, Shiqian Ma, Lingzhou Xue

Riemannian optimization has drawn a lot of attention due to its wide applications in practice. Riemannian stochastic first-order algorithms have been studied in the literature to solve large-scale machine learning proble…

Low-Rank Matrix Completion Matrix Completion Riemannian optimization

The Distributionally Robust Optimization Model of Sparse Principal Component Analysis

2025-03-04 · Lei Wang, Xin Liu, Xiaojun Chen

We consider sparse principal component analysis (PCA) under a stochastic setting where the underlying probability distribution of the random parameter is uncertain. This problem is formulated as a distributionally robust…

Riemannian optimization

Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods

2019-11-12 · Xiao Li, Shixiang Chen, Zengde Deng, Qing Qu 외

We consider a class of nonsmooth optimization problems over the Stiefel manifold, in which the objective function is weakly convex in the ambient Euclidean space. Such problems are ubiquitous in engineering applications …

Dictionary Learning Vocal Bursts Type Prediction

A Manifold Proximal Linear Method for Sparse Spectral Clustering with Application to Single-Cell RNA Sequencing Data Analysis

2020-07-18 · Zhongruo Wang, Bingyuan Liu, Shixiang Chen, Shiqian Ma 외

Spectral clustering is one of the fundamental unsupervised learning methods widely used in data analysis. Sparse spectral clustering (SSC) imposes sparsity to the spectral clustering and it improves the interpretability …

Clustering

Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization

2016-12-01 · NeurIPS 2016 12 · Sashank J. Reddi, Suvrit Sra, Barnabas Poczos, Alexander J. Smola

We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem is very limit…