Regularized Estimation and Feature Selection in Mixtures of Gaussian-Gated Experts Models
Mixtures-of-Experts models and their maximum likelihood estimation (MLE) via the EM algorithm have been thoroughly studied in the statistics and machine learning literature. They are subject of a growing investigation in the context of modeling with high-dimensional predictors with regularized MLE. We examine MoE with Gaussian gating network, for clustering and regression, and propose an $\ell_1$-regularized MLE to encourage sparse models and deal with the high-dimensional setting. We develop an EM-Lasso algorithm to perform parameter estimation and utilize a BIC-like criterion to select the model parameters, including the sparsity tuning hyperparameters. Experiments conducted on simulated data show the good performance of the proposed regularized MLE compared to the standard MLE with the EM algorithm.
Code (0)
등록된 구현이 없습니다.
Tasks
Clusteringfeature selectionparameter estimationregressionSimilar Papers 제목 키워드 기반
Estimation and Feature Selection in Mixtures of Generalized Linear Experts Models
Mixtures-of-Experts (MoE) are conditional mixture models that have shown their performance in modeling heterogeneity in data in many statistical learning approaches for prediction, including regression and classification…
Clusteringfeature selectionparameter estimationregressionRegularized Maximum Likelihood Estimation and Feature Selection in Mixtures-of-Experts Models
Mixture of Experts (MoE) are successful models for modeling heterogeneous data in many statistical learning problems including regression, clustering and classification. Generally fitted by maximum likelihood estimation …
Clusteringfeature selectionMixture-of-Expertsparameter estimation+1A MAP approach for $\ell_q$-norm regularized sparse parameter estimation using the EM algorithm
In this paper, Bayesian parameter estimation through the consideration of the Maximum A Posteriori (MAP) criterion is revisited under the prism of the Expectation-Maximization (EM) algorithm. By incorporating a sparsity-…
parameter estimationOptimal Transport Tools (OTT): A JAX Toolbox for all things Wasserstein
Optimal transport tools (OTT-JAX) is a Python toolbox that can solve optimal transport problems between point clouds and histograms. The toolbox builds on various JAX features, such as automatic and custom reverse mode d…
AllAccelerated Regularized Wasserstein Proximal Sampling Algorithms
We consider sampling from a Gibbs distribution by evolving a finite number of particles using a particular score estimator rather than Brownian motion. To accelerate the particles, we consider a second-order score-based …