Latent Coincidence Analysis: A Hidden Variable Model for Distance Metric Learning
We describe a latent variable model for supervised dimensionality reduction and distance metric learning. The model discovers linear projections of high dimensional data that shrink the distance between similarly labeled inputs and expand the distance between differently labeled ones. The model’s continuous latent variables locate pairs of examples in a latent space of lower dimensionality. The model differs significantly from classical factor analysis in that the posterior distribution over these latent variables is not always multivariate Gaussian. Nevertheless we show that inference is completely tractable and derive an Expectation-Maximization (EM) algorithm for parameter estimation. We also compare the model to other approaches in distance metric learning. The model’s main advantage is its simplicity: at each iteration of the EM algorithm, the distance metric is re-estimated by solving an unconstrained least-squares problem. Experiments show that these simple updates are highly effective.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionMetric Learningparameter estimationSupervised dimensionality reductionSimilar Papers 제목 키워드 기반
Bayesian Estimation of Multidimensional Latent Variables and Its Asymptotic Accuracy
Hierarchical learning models, such as mixture models and Bayesian networks, are widely employed for unsupervised learning tasks, such as clustering analysis. They consist of observable and hidden variables, which represe…
ClusteringDiscovering Hidden Variables in Noisy-Or Networks using Quartet Tests
We give a polynomial-time algorithm for provably learning the structure and parameters of bipartite noisy-or Bayesian networks of binary variables where the top layer is completely hidden. Unsupervised learning of these …
As Easy as ABC: Adaptive Binning Coincidence Test for Uniformity Testing
We consider the problem of uniformity testing of Lipschitz continuous distributions with bounded support. The alternative hypothesis is a composite set of Lipschitz continuous distributions that are at least $\varepsilon…
Particle Dynamics for Latent-Variable Energy-Based Models
Latent-variable energy-based models (LVEBMs) assign a single normalized energy to joint pairs of observed data and latent variables, offering expressive generative modeling while capturing hidden structure. We recast max…
Learning and Understanding a Disentangled Feature Representation for Hidden Parameters in Reinforcement Learning
Hidden parameters are latent variables in reinforcement learning (RL) environments that are constant over the course of a trajectory. Understanding what, if any, hidden parameters affect a particular environment can aid …
Metric Learningreinforcement-learningReinforcement Learning (RL)