Generative Local Metric Learning for Nearest Neighbor Classification
We consider the problem of learning a local metric to enhance the performance of nearest neighbor classification. Conventional metric learning methods attempt to separate data distributions in a purely discriminative manner; here we show how to take advantage of information from parametric generative models. We focus on the bias in the information-theoretic error arising from finite sampling effects, and find an appropriate local metric that maximally reduces the bias based upon knowledge from generative models. As a byproduct, the asymptotic theoretical analysis in this work relates metric learning with dimensionality reduction, which was not understood from previous discriminative approaches. Empirical experiments show that this learned local metric enhances the discriminative nearest neighbor performance on various datasets using simple class conditional generative models.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationDimensionality ReductionGeneral ClassificationMetric LearningSimilar Papers 제목 키워드 기반
A Two-Stage Active Learning Algorithm for $k$-Nearest Neighbors
$k$-nearest neighbor classification is a popular non-parametric method because of desirable properties like automatic adaption to distributional scale changes. Unfortunately, it has thus far proved difficult to design ac…
Active LearningVocal Bursts Valence PredictionA Novel Pseudo Nearest Neighbor Classification Method Using Local Harmonic Mean Distance
In the realm of machine learning, the KNN classification algorithm is widely recognized for its simplicity and efficiency. However, its sensitivity to the K value poses challenges, especially with small sample sizes or o…
ClassificationSensitivityParametric Local Metric Learning for Nearest Neighbor Classification
We study the problem of learning local metrics for nearest neighbor classification. Most previous works on local metric learning learn a number of local unrelated metrics. While this ''independence'' approach delivers an…
ClassificationGeneral ClassificationMetric LearningRates of Convergence for Nearest Neighbor Classification
Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates f…
ClassificationGeneral ClassificationMulti-View Semi-Supervised Label Distribution Learning with Local Structure Complementarity
Label distribution learning (LDL) is a paradigm that each sample is associated with a label distribution. At present, the existing approaches are proposed for the single-view LDL problem with labeled data, while the mult…
Graph Learning