paper-with-me

홈 › Papers

Global Convergence of EM Algorithm for Mixtures of Two Component Linear Regression

2018-10-12 · Jeongyeol Kwon, Wei Qian, Constantine Caramanis, Yudong Chen, Damek Davis

The Expectation-Maximization algorithm is perhaps the most broadly used algorithm for inference of latent variable problems. A theoretical understanding of its performance, however, largely remains lacking. Recent results established that EM enjoys global convergence for Gaussian Mixture Models. For Mixed Linear Regression, however, only local convergence results have been established, and those only for the high SNR regime. We show here that EM converges for mixed linear regression with two components (it is known that it may fail to converge for three or more), and moreover that this convergence holds for random initialization. Our analysis reveals that EM exhibits very different behavior in Mixed Linear Regression from its behavior in Gaussian Mixture Models, and hence our proofs require the development of several new ideas.

📄 PDF Abstract BibTeX arXiv:1810.05752

Code (0)

등록된 구현이 없습니다.

Tasks

regression

Methods 이 논문이 사용한 방법론

Linear Regression Linear Regression is a method for modelling a relationship between a dependent variable and independent variables. These models can be fit with numerous approaches. The most…

Similar Papers 제목 키워드 기반

Toward Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixture Models

2024-06-29 · Weihang Xu, Maryam Fazel, Simon S. Du

We study the gradient Expectation-Maximization (EM) algorithm for Gaussian Mixture Models (GMM) in the over-parameterized setting, where a general GMM with $n>1$ components learns from data that are generated by a single…

Learning Mixtures of Linear Regressions with Nearly Optimal Complexity

2018-02-22 · Yuanzhi Li, YIngyu Liang

Mixtures of Linear Regressions (MLR) is an important mixture model with many applications. In this model, each observation is generated from one of the several unknown linear regression components, where the identity of …

Gradient Algorithms for Complex Non-Gaussian Independent Component/Vector Extraction, Question of Convergence

2018-06-26

We revise the problem of extracting one independent component from an instantaneous linear mixture of signals. The mixing matrix is parameterized by two vectors, one column of the mixing matrix and one row of the de-mixi…

Dynamic Independent Component/Vector Analysis: Time-Variant Linear Mixtures Separable by Time-Invariant Beamformers

2020-07-22 · Zbyněk Koldovský, Václav Kautský, Petr Tichavský

A novel extension of Independent Component and Independent Vector Analysis for blind extraction/separation of one or several sources from time-varying mixtures is proposed. The mixtures are assumed to be separable source…

Learning Mixtures of Experts with EM

2024-11-09 · Quentin Fruytier, Aryan Mokhtari, Sujay Sanghavi

Mixtures of Experts (MoE) are Machine Learning models that involve partitioning the input space, with a separate "expert" model trained on each partition. Recently, MoE have become popular as components in today's large …