Vector Embeddings with Subvector Permutation Invariance using a Triplet Enhanced Autoencoder
The use of deep neural network (DNN) autoencoders (AEs) has recently exploded due to their wide applicability. However, the embedding representation produced by a standard DNN AE that is trained to minimize only the reconstruction error does not always reveal more subtle patterns in the data. Sometimes, the autoencoder needs further direction in the form of one or more additional loss functions. In this paper, we use an autoencoder enhanced with triplet loss to promote the clustering of vectors that are related through permutations of constituent subvectors. With this approach, we can create an embedding of the vector that is nearly invariant to such permutations. We can then use these invariant embeddings as inputs to other problems, like classification and clustering, and improve detection accuracy in those problems.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringTripletMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Robust Permutation Test for Subvector Inference in Linear Regressions
We develop a new permutation test for inference on a subvector of coefficients in linear models. The test is exact when the regressors and the error terms are independent. Then, we show that the test is asymptotically of…
Semantics- and Syntax-related Subvectors in the Skip-gram Embeddings
We show that the skip-gram embedding of any word can be decomposed into two subvectors which roughly correspond to semantic and syntactic roles of the word.
Permutation invariant functions: statistical tests, density estimation, and computationally efficient embedding
Permutation invariance is among the most common symmetry that can be exploited to simplify complex problems in machine learning (ML). There has been a tremendous surge of research activities in building permutation invar…
Density EstimationDimensionality ReductionGraViti: Graph-Level Variational Autoencoders with Relaxed Permutation Invariance
We introduce GraViti, a transformer-based graph-level variational autoencoder that maps entire graphs to compact latent vectors. This design produces a true graph-level latent space that supports smooth interpolation, pr…
Bringing Order to Neural Word Embeddings with Embeddings Augmented by Random Permutations (EARP)
Word order is clearly a vital part of human language, but it has been used comparatively lightly in distributional vector models. This paper presents a new method for incorporating word order information into word vector…
RetrievalWord Embeddings