Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model
A significant roadblock in multilingual neural language modeling is the lack of labeled non-English data. One potential method for overcoming this issue is learning cross-lingual text representations that can be used to transfer the performance from training on English tasks to non-English tasks, despite little to no task-specific non-English data. In this paper, we explore a natural setup for learning cross-lingual sentence representations: the dual-encoder. We provide a comprehensive evaluation of our cross-lingual representations on a number of monolingual, cross-lingual, and zero-shot/few-shot learning tasks, and also give an analysis of different learned cross-lingual embedding spaces.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningLanguage ModelingLanguage ModellingSentenceSimilar Papers 제목 키워드 기반
Probing Multilingual Sentence Representations With X-Probe
This paper extends the task of probing sentence representations for linguistic insight in a multilingual domain. In doing so, we make two contributions: first, we provide datasets for multilingual probing, derived from W…
Natural Language InferenceSentenceA Multi-task Approach to Learning Multilingual Representations
We present a novel multi-task modeling approach to learning multilingual distributed representations of text. Our system learns word and sentence embeddings jointly by training a multilingual skip-gram model together wit…
Cross-Lingual Document ClassificationDocument ClassificationGeneral ClassificationSentence+3Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization
Multilingual sentence representations are the foundation for similarity-based bitext mining, which is crucial for scaling multilingual neural machine translation (NMT) system to more languages. In this paper, we introduc…
DecoderMachine TranslationNMTSentence+1Lightweight Cross-Lingual Sentence Representation Learning
Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER (Artetxe and Schwenk, 2019b) lead to significant improvement in performance on downstream tasks. However, further increa…
Contrastive LearningDocument ClassificationLanguage ModelingLanguage Modelling+4Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity
Previous work has shown that the representations output by contextual language models are more anisotropic than static type embeddings, and typically display outlier dimensions. This seems to be true for both monolingual…
Semantic SimilaritySemantic Textual SimilaritySentenceSentence Similarity