Unsupervised Cross-Lingual Representation Learning
In this tutorial, we provide a comprehensive survey of the exciting recent work on cutting-edge weakly-supervised and unsupervised cross-lingual word representations. After providing a brief history of supervised cross-lingual word representations, we focus on: 1) how to induce weakly-supervised and unsupervised cross-lingual word representations in truly resource-poor settings where bilingual supervision cannot be guaranteed; 2) critical examinations of different training conditions and requirements under which unsupervised algorithms can and cannot work effectively; 3) more robust methods for distant language pairs that can mitigate instability issues and low performance for distant language pairs; 4) how to comprehensively evaluate such representations; and 5) diverse applications that benefit from cross-lingual word representations (e.g., MT, dialogue, cross-lingual sequence labeling and structured prediction applications, cross-lingual IR).
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningStructured PredictionSimilar Papers 제목 키워드 기반
Cross-lingual Feature Extraction from Monolingual Corpora for Low-resource Unsupervised Bilingual Lexicon Induction
Despite their progress in high-resource language settings, unsupervised bilingual lexicon induction (UBLI) models often fail on corpora with low-resource distant language pairs due to insufficient initialization. In this…
Bilingual Lexicon InductionWord EmbeddingsBilingual alignment transfers to multilingual alignment for unsupervised parallel text mining
This work presents methods for learning cross-lingual sentence representations using paired or unpaired bilingual texts. We hypothesize that the cross-lingual alignment strategy is transferable, and therefore a model tra…
Language ModelingLanguage ModellingRetrievalSentence+1Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining
Existing models of multilingual sentence embeddings require large parallel data resources which are not available for low-resource languages. We propose a novel unsupervised method to derive multilingual sentence embeddi…
Language ModelingLanguage ModellingMachine TranslationParallel Corpus Mining+4Fully Unsupervised Crosslingual Semantic Textual Similarity Metric Based on BERT for Identifying Parallel Data
We present a fully unsupervised crosslingual semantic textual similarity (STS) metric, based on contextual embeddings extracted from BERT {--} Bidirectional Encoder Representations from Transformers (Devlin et al., 2019)…
Machine TranslationNatural Language UnderstandingSemantic Textual SimilaritySTS+1Unsupervised Geometric and Topological Approaches for Cross-Lingual Sentence Representation and Comparison
We propose novel structural-based approaches for the generation and comparison of cross lingual sentence representations. We do so by applying geometric and topological methods to analyze the structure of sentences, as c…
SentenceWord Embeddings