NMT-based Cross-lingual Document Embeddings
This paper investigates a cross-lingual document embedding method that improves the current Neural machine Translation framework based Document Vector (NTDV or simply NV). NV is developed with a self-attention mechanism under the neural machine translation (NMT) framework. In NV, each pair of parallel documents in different languages are projected to the same shared layer in the model. However, the pair of NV embeddings are not guaranteed to be similar. This paper further adds a distance constraint to the training objective function of NV so that the two embeddings of a parallel document are required to be as close as possible. The new method will be called constrained NV (cNV). In a cross-lingual document classification task, the new cNV performs as well as NV and outperforms other published studies that require forward-pass decoding. Compared with the previous NV, cNV does not need a translator during testing, and so the method is lighter and more flexible.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual Document ClassificationDocument ClassificationDocument EmbeddingGeneral ClassificationMachine TranslationNMTTranslationSimilar Papers 제목 키워드 기반
Multilingual Training of Crosslingual Word Embeddings
Crosslingual word embeddings represent lexical items from different languages using the same vector space, enabling crosslingual transfer. Most prior work constructs embeddings for a pair of languages, with English on on…
Bilingual Lexicon InductionDependency ParsingDocument ClassificationGeneral Classification+6Joint Training for Learning Cross-lingual Embeddings with Sub-word Information without Parallel Corpora
In this paper, we propose a novel method for learning cross-lingual word embeddings, that incorporates sub-word information during training, and is able to learn high-quality embeddings from modest amounts of monolingual…
Bilingual Lexicon InductionClassificationCross-Lingual Word EmbeddingsDocument Classification+2Wasserstein distances for evaluating cross-lingual embeddings
Word embeddings are high dimensional vector representations of words that capture their semantic similarity in the vector space. There exist several algorithms for learning such embeddings both for a single language as w…
Cross-Lingual Document ClassificationDocument ClassificationRetrievalSemantic Similarity+2A Comparison of Unsupervised Methods for Ad hoc Cross-Lingual Document Retrieval
We address the problem of linking related documents across languages in a multilingual collection. We evaluate three diverse unsupervised methods to represent and compare documents: (1) multilingual topic model; (2) cros…
ArticlesRetrievalA Bayesian Multilingual Document Model for Zero-shot Topic Identification and Discovery
In this paper, we present a Bayesian multilingual document model for learning language-independent document embeddings. The model is an extension of BaySMM [Kesiraju et al 2020] to the multilingual scenario. It learns to…
SentenceXLM-R