Derivation of Learning Rules for Coupled Principal Component Analysis in a Lagrange-Newton Framework
We describe a Lagrange-Newton framework for the derivation of learning rules with desirable convergence properties and apply it to the case of principal component analysis (PCA). In this framework, a Newton descent is applied to an extended variable vector which also includes Lagrange multipliers introduced with constraints. The Newton descent guarantees equal convergence speed from all directions, but is also required to produce stable fixed points in the system with the extended state vector. The framework produces "coupled" PCA learning rules which simultaneously estimate an eigenvector and the corresponding eigenvalue in cross-coupled differential equations. We demonstrate the feasibility of this approach for two PCA learning rules, one for the estimation of the principal, the other for the estimate of an arbitrary eigenvector-eigenvalue pair (eigenpair).
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Derivation of Coupled PCA and SVD Learning Rules from a Newton Zero-Finding Framework
In coupled learning rules for PCA (principal component analysis) and SVD (singular value decomposition), the update of the estimates of eigenvectors or singular vectors is influenced by the estimates of eigenvalues or si…
Derivation of Symmetric PCA Learning Rules from a Novel Objective Function
Neural learning rules for principal component / subspace analysis (PCA / PSA) can be derived by maximizing an objective function (summed variance of the projection on the subspace axes) under an orthonormality constraint…
A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
Neural network models of early sensory processing typically reduce the dimensionality of streaming input data. Such networks learn the principal subspace, in the sense of principal component analysis (PCA), by adjusting …
Generalized Principal Component Analysis
Generalized principal component analysis (GLM-PCA) facilitates dimension reduction of non-normally distributed data. We provide a detailed derivation of GLM-PCA with a focus on optimization. We also demonstrate how to in…
Dimensionality ReductionTwo derivations of Principal Component Analysis on datasets of distributions
In this brief note, we formulate Principal Component Analysis (PCA) over datasets consisting not of points but of distributions, characterized by their location and covariance. Just like the usual PCA on points can be eq…