Learning Mixtures of Low-Rank Models
We study the problem of learning mixtures of low-rank models, i.e. reconstructing multiple low-rank matrices from unlabelled linear measurements of each. This problem enriches two widely studied settings -- low-rank matrix sensing and mixed linear regression -- by bringing latent variables (i.e. unknown labels) and structural priors (i.e. low-rank structures) into consideration. To cope with the non-convexity issues arising from unlabelled heterogeneous data and low-complexity structure, we develop a three-stage meta-algorithm that is guaranteed to recover the unknown matrices with near-optimal sample and computational complexities under Gaussian designs. In addition, the proposed algorithm is provably stable against random noise. We complement the theoretical studies with empirical evidence that confirms the efficacy of our algorithm.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On the identifiability of mixtures of ranking models
Mixtures of ranking models are standard tools for ranking problems. However, even the fundamental question of parameter identifiability is not fully understood: the identifiability of a mixture model with two Bradley-Ter…
Learning Mixtures of Plackett-Luce Models from Structured Partial Orders
Mixtures of ranking models have been widely used for heterogeneous preferences. However, learning a mixture model is highly nontrivial, especially when the dataset consists of partial orders. In such cases, the parameter…
2kComputational EfficiencyInducing Generalization across Languages and Tasks using Featurized Low-Rank Mixtures
Adapting pretrained large language models (LLMs) to various downstream tasks in tens or hundreds of human languages is computationally expensive. Parameter-efficient fine-tuning (PEFT) significantly reduces the adaptatio…
parameter-efficient fine-tuningProbabilistic risk aversion for generalized rank-dependent functions
Probabilistic risk aversion, defined through quasi-convexity in probabilistic mixtures, is a common useful property in decision analysis. We study a general class of non-monotone mappings, called the generalized rank-dep…
ManagementIdentifiability and Estimation for Unlabeled Finite Mixtures under Marginal Independence
We study component recovery and mixing-matrix estimation from unlabeled finite mixtures whose observable distributions share the same latent components but have unknown mixing weights. The main identifying signal is marg…