A Multi-task Approach to Learning Multilingual Representations
We present a novel multi-task modeling approach to learning multilingual distributed representations of text. Our system learns word and sentence embeddings jointly by training a multilingual skip-gram model together with a cross-lingual sentence similarity model. Our architecture can transparently use both monolingual and sentence aligned bilingual corpora to learn multilingual embeddings, thus covering a vocabulary significantly larger than the vocabulary of the bilingual corpora alone. Our model shows competitive performance in a standard cross-lingual document classification task. We also show the effectiveness of our method in a limited resource scenario.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual Document ClassificationDocument ClassificationGeneral ClassificationSentenceSentence EmbeddingsSentence SimilarityWord EmbeddingsSimilar Papers 제목 키워드 기반
Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity
Previous work has shown that the representations output by contextual language models are more anisotropic than static type embeddings, and typically display outlier dimensions. This seems to be true for both monolingual…
Semantic SimilaritySemantic Textual SimilaritySentenceSentence SimilarityNeural Architectures for Multilingual Semantic Parsing
In this paper, we address semantic parsing in a multilingual context. We train one multilingual model that is capable of parsing natural language sentences from multiple different languages into their corresponding forma…
DecoderMachine TranslationMulti-Task LearningSemantic ParsingAssessing Multilingual Fairness in Pre-trained Multimodal Representations
Recently pre-trained multimodal models, such as CLIP, have shown exceptional capabilities towards connecting images and natural language. The textual representations in English can be desirably transferred to multilingua…
FairnessCode-switched inspired losses for spoken dialog representations
Spoken dialogue systems need to be able to handle both multiple languages and multilinguality inside a conversation (e.g in case of code-switching). In this work, we introduce new pretraining losses tailored to learn gen…
RetrievalSpoken Dialogue SystemsCode-switched inspired losses for generic spoken dialog representations
Spoken dialog systems need to be able to handle both multiple languages and multilinguality inside a conversation (\textit{e.g} in case of code-switching). In this work, we introduce new pretraining losses tailored to le…
Retrieval