Locality Preserving Loss: Neighbors that Live together, Align together
We present a locality preserving loss (LPL) that improves the alignment between vector space embeddings while separating uncorrelated representations. Given two pretrained embedding manifolds, LPL optimizes a model to project an embedding and maintain its local neighborhood while aligning one manifold to another. This reduces the overall size of the dataset required to align the two in tasks such as cross-lingual word alignment. We show that the LPL-based alignment between input vector spaces acts as a regularizer, leading to better and consistent accuracy than the baseline, especially when the size of the training set is small. We demonstrate the effectiveness of LPL optimized alignment on semantic text similarity (STS), natural language inference (SNLI), multi-genre language inference (MNLI) and cross-lingual word alignment(CLA) showing consistent improvements, finding up to 16% improvement over our baseline in lower resource settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceSentence EmbeddingsSTStext similarityWord AlignmentSimilar Papers 제목 키워드 기반
Locality Sensitive Hashing for Set-Queries, Motivated by Group Recommendations
Locality Sensitive Hashing (LSH) is an effective method to index a set of points such that we can efficiently find the nearest neighbors of a query point. We extend this method to our novel Set-query LSH (SLSH), such tha…
Recommendation SystemsLocality-Sensitive Hashing for f-Divergences: Mutual Information Loss and Beyond
Computing approximate nearest neighbors in high dimensional spaces is a central problem in large-scale data mining with a wide range of applications in machine learning and data science. A popular and effective technique…
Model CompressionLocal Shapley: Model-Induced Locality and Optimal Reuse in Data Valuation
The Shapley value provides a principled foundation for data valuation, but exact computation is #P-hard due to the exponential coalition space. Existing accelerations remain global and ignore a structural property of mod…
Local Low-Rank Approximation With Superpixel-Guided Locality Preserving Graph for Hyperspectral Image Classification
Given the detrimental effect of spectral variations in a hyperspectral image (HSI), this article investigates to recover its discriminative representation to improve the classification performance. We propose a new metho…
Hyperspectral Image Classificationimage-classificationImage ClassificationSuperpixelsSub-Linear Privacy-Preserving Near-Neighbor Search
In Near-Neighbor Search (NNS), a new client queries a database (held by a server) for the most similar data (near-neighbors) given a certain similarity metric. The Privacy-Preserving variant (PP-NNS) requires that neithe…
Privacy Preserving