Robust High Dimensional Expectation Maximization Algorithm via Trimmed Hard Thresholding
In this paper, we study the problem of estimating latent variable models with arbitrarily corrupted samples in high dimensional space ({\em i.e.,} $d\gg n$) where the underlying parameter is assumed to be sparse. Specifically, we propose a method called Trimmed (Gradient) Expectation Maximization which adds a trimming gradients step and a hard thresholding step to the Expectation step (E-step) and the Maximization step (M-step), respectively. We show that under some mild assumptions and with an appropriate initialization, the algorithm is corruption-proofing and converges to the (near) optimal statistical rate geometrically when the fraction of the corrupted samples $\epsilon$ is bounded by $ \tilde{O}(\frac{1}{\sqrt{n}})$. Moreover, we apply our general framework to three canonical models: mixture of Gaussians, mixture of regressions and linear regression with missing covariates. Our theory is supported by thorough numerical results.
Code (0)
등록된 구현이 없습니다.
Tasks
Vocal Bursts Intensity PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normality
We provide a general theory of the expectation-maximization (EM) algorithm for inferring high dimensional latent variable models. In particular, we make two contributions: (i) For parameter estimation, we propose a novel…
parameter estimationVocal Bursts Intensity PredictionHigh-Dimensional Variance-Reduced Stochastic Gradient Expectation-Maximization Algorithm
We propose a generic stochastic expectation-maximization (EM) algorithm for the estimation of high-dimensional latent variable models. At the core of our algorithm is a novel semi-stochastic variance-reduced gradien…
parameter estimationVocal Bursts Intensity PredictionComponent-wise Adaptive Trimming For Robust Mixture Regression
Parameter estimation of mixture regression model using the expectation maximization (EM) algorithm is highly sensitive to outliers. Here we propose a fast and efficient robust mixture regression algorithm, called Compone…
Outlier Detectionparameter estimationregressionStochastic First-Order Learning for Large-Scale Flexibly Tied Gaussian Mixture Model
Gaussian Mixture Models (GMMs) are one of the most potent parametric density models used extensively in many applications. Flexibly-tied factorization of the covariance matrices in GMMs is a powerful approach for coping …
Stochastic OptimizationNoise Benefits in Expectation-Maximization Algorithms
This dissertation shows that careful injection of noise into sample data can substantially speed up Expectation-Maximization algorithms. Expectation-Maximization algorithms are a class of iterative algorithms for extract…
Bayesian InferenceClusteringparameter estimation