Supervised Exponential Family Principal Component Analysis via Convex Optimization
Recently, supervised dimensionality reduction has been gaining attention, owing to the realization that data labels are often available and strongly suggest important underlying structures in the data. In this paper, we present a novel convex supervised dimensionality reduction approach based on exponential family PCA and provide a simple but novel form to project new testing data into the embedded space. This convex approach successfully avoids the local optima of the EM learning. Moreover, by introducing a sample-based multinomial approximation to exponential family models, it avoids the limitation of the prevailing Gaussian assumptions of standard PCA, and produces a kernelized formulation for nonlinear supervised dimensionality reduction. A training algorithm is then devised based on a subgradient bundle method, whose scalability can be gained through a coordinate descent procedure. The advantage of our global optimization approach is demonstrated by empirical results over both synthetic and real data.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality Reductionglobal-optimizationSupervised dimensionality reductionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
$e^{\text{RPCA}}$: Robust Principal Component Analysis for Exponential Family Distributions
Robust Principal Component Analysis (RPCA) is a widely used method for recovering low-rank structure from data matrices corrupted by significant and sparse outliers. These corruptions may arise from occlusions, malicious…
Defect DetectionIterated and exponentially weighted moving principal component analysis
The principal component analysis (PCA) is a staple statistical and unsupervised machine learning technique in finance. The application of PCA in a financial setting is associated with several technical difficulties, such…
Deep equilibrium models as estimators for continuous latent variables
Principal Component Analysis (PCA) and its exponential family extensions have three components: observations, latents and parameters of a linear transformation. We consider a generalised setting where the canonical param…
Bayesian Exponential Family PCA
Principal Components Analysis (PCA) has become established as one of the key tools for dimensionality reduction when dealing with real valued data. Approaches such as exponential family PCA and non-negative matrix factor…
Bayesian InferenceDimensionality ReductionExplaining and Generalizing Skip-Gram through Exponential Family Principal Component Analysis
The popular skip-gram model induces word embeddings by exploiting the signal from word-context coocurrence. We offer a new interpretation of skip-gram based on exponential family PCA-a form of matrix factorization to gen…
Word Embeddings