Online Partial Least Square Optimization: Dropping Convexity for Better Efficiency and Scalability
Multiview representation learning is popular for latent factor analysis. Many existing approaches formulate the multiview representation learning as convex optimization problems, where global optima can be obtained by certain algorithms in polynomial time. However, many evidences have corroborated that heuristic nonconvex approaches also have good empirical computational performance and convergence to the global optima, although there is a lack of theoretical justification. Such a gap between theory and practice motivates us to study a nonconvex formulation for multiview representation learning, which can be efficiently solved by a simple stochastic gradient descent method. By analyzing the dynamics of the algorithm based on diffusion processes, we establish a global rate of convergence to the global optima. Numerical experiments are provided to support our theory.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningSimilar Papers 제목 키워드 기반
Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
The problem of solving partial differential equations (PDEs) can be formulated into a least-squares minimization problem, where neural networks are used to parametrize PDE solutions. A global minimizer corresponds to a n…
Covariance-free Partial Least Squares: An Incremental Dimensionality Reduction Method
Dimensionality reduction plays an important role in computer vision problems since it reduces computational cost and is often capable of yielding more discriminative data representation. In this context, Partial Least Sq…
Computational EfficiencyDimensionality ReductionFace Verificationfeature selection+3Robust Partially-Compressed Least-Squares
Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency…
Computational EfficiencyDAPLSR: Data Augmentation Partial Least Squares Regression Model via Manifold Optimization
Traditional Partial Least Squares Regression (PLSR) models frequently underperform when handling data characterized by uneven categories. To address the issue, this paper proposes a Data Augmentation Partial Least Square…
Data AugmentationregressionPartial Least Square Regression via Three-factor SVD-type Manifold Optimization for EEG Decoding
Partial least square regression (PLSR) is a widely-used statistical model to reveal the linear relationships of latent factors that comes from the independent variables and dependent variables. However, traditional metho…
EEGEeg DecodingElectroencephalogram (EEG)Motor Imagery+2