Context-Aware Cross-Lingual Mapping
Cross-lingual word vectors are typically obtained by fitting an orthogonal matrix that maps the entries of a bilingual dictionary from a source to a target vector space. Word vectors, however, are most commonly used for sentence or document-level representations that are calculated as the weighted average of word embeddings. In this paper, we propose an alternative to word-level mapping that better reflects sentence-level cross-lingual similarity. We incorporate context in the transformation matrix by directly mapping the averaged embeddings of aligned sentences in a parallel corpus. We also implement cross-lingual mapping of deep contextualized word embeddings using parallel sentences with word alignments. In our experiments, both approaches resulted in cross-lingual sentence embeddings that outperformed context-independent word mapping in sentence translation retrieval. Furthermore, the sentence-level transformation could be used for word-level mapping without loss in word translation quality.
Code (1)
Tasks
RetrievalSentenceSentence EmbeddingsTranslationWord EmbeddingsWord TranslationSimilar Papers 제목 키워드 기반
Cross-Lingual BERT Contextual Embedding Space Mapping with Isotropic and Isometric Conditions
Typically, a linearly orthogonal transformation mapping is learned by aligning static type-level embeddings to build a shared semantic space. In view of the analysis that contextual embeddings contain richer semantic fea…
Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing
We introduce a novel method for multilingual transfer that utilizes deep contextual embeddings, pretrained in an unsupervised fashion. While contextual embeddings have been shown to yield richer representations of meanin…
Cross-lingual zero-shot dependency parsingDependency ParsingFew-Shot LearningWord EmbeddingsCross-lingual alignments of ELMo contextual embeddings
Building machine learning prediction models for a specific NLP task requires sufficient training data, which can be difficult to obtain for less-resourced languages. Cross-lingual embeddings map word embeddings from a le…
Dependency Parsingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+3Revisiting the Context Window for Cross-lingual Word Embeddings
Existing approaches to mapping-based cross-lingual word embeddings are based on the assumption that the source and target embedding spaces are structurally similar. The structures of embedding spaces largely depend on th…
Bilingual Lexicon InductionCross-Lingual Word EmbeddingsWord EmbeddingsZero-Shot Cross-Lingual Dependency Parsing through Contextual Embedding Transformation
Linear embedding transformation has been shown to be effective for zero-shot cross-lingual transfer tasks and achieve surprisingly promising results. However, cross-lingual embedding space mapping is usually studied in s…
Cross-Lingual TransferDependency ParsingTranslationZero-Shot Cross-Lingual Transfer