It’s not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT
Recent works have demonstrated that multilingual BERT (mBERT) learns rich cross-lingual representations, that allow for transfer across languages. We study the word-level translation information embedded in mBERT and present two simple methods that expose remarkable translation capabilities with no fine-tuning. The results suggest that most of this information is encoded in a non-linear way, while some of it can also be recovered with purely linear tools. As part of our analysis, we test the hypothesis that mBERT learns representations which contain both a language-encoding component and an abstract, cross-lingual component, and explicitly identify an empirical language-identity subspace within mBERT representations.
Code (1)
Tasks
TranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT
Recent works have demonstrated that multilingual BERT (mBERT) learns rich cross-lingual representations, that allow for transfer across languages. We study the word-level translation information embedded in mBERT and pre…
TranslationAncient Greek WordNet Meets the Dynamic Lexicon: the Example of the Fragments of the Greek Historians
The Ancient Greek WordNet (AGWN) and the Dynamic Lexicon (DL) are multilingual resources to study the lexicon of Ancient Greek texts and their translations. Both AGWN and DL are works in progress that need accuracy impro…
GLaM-Sign: Greek Language Multimodal Lip Reading with Integrated Sign Language Accessibility
The Greek Language Multimodal Lip Reading with Integrated Sign Language Accessibility (GLaM-Sign) [1] is a groundbreaking resource in accessibility and multimodal AI, designed to support Deaf and Hard-of-Hearing (DHH) in…
Lip ReadingSign Language TranslationEnsemble Fine-tuned mBERT for Translation Quality Estimation
Quality Estimation (QE) is an important component of the machine translation workflow as it assesses the quality of the translated output without consulting reference translations. In this paper, we discuss our submissio…
Machine TranslationSentenceTask 2TranslationPENELOPIE: Enabling Open Information Extraction for the Greek Language through Machine Translation
In this paper we present our submission for the EACL 2021 SRW; a methodology that aims at bridging the gap between high and low-resource languages in the context of Open Information Extraction, showcasing it on the Greek…
Machine TranslationNMTOpen Information ExtractionTranslation