Low-Resource Unsupervised NMT: Diagnosing the Problem and Providing a Linguistically Motivated Solution
Unsupervised Machine Translation has been advancing our ability to translate without parallel data, but state-of-the-art methods assume an abundance of monolingual data. This paper investigates the scenario where monolingual data is limited as well, finding that current unsupervised methods suffer in performance under this stricter setting. We find that the performance loss originates from the poor quality of the pretrained monolingual embeddings, and we offer a potential solution: dependency-based word embeddings. These embeddings result in a complementary word representation which offers a boost in performance of around 1.5 BLEU points compared to standard word2vec when monolingual data is limited to 1 million sentences per language. We also find that the inclusion of sub-word information is crucial to improving the quality of the embeddings.
Code (1)
Tasks
Machine TranslationNMTTranslationUnsupervised Machine TranslationWord EmbeddingsSimilar Papers 제목 키워드 기반
Exploiting Cross-Lingual Knowledge in Unsupervised Acoustic Modeling for Low-Resource Languages
(Short version of Abstract) This thesis describes an investigation on unsupervised acoustic modeling (UAM) for automatic speech recognition (ASR) in the zero-resource scenario, where only untranscribed speech data is ass…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language Acquisitionspeech-recognition+1The SETimes.HR Linguistically Annotated Corpus of Croatian
We present SETimes.HR ― the first linguistically annotated corpus of Croatian that is freely available for all purposes. The corpus is built on top of the SETimes parallel corpus of nine Southeast European languages an…
AllBoundary DetectionDependency ParsingLemmatization+4A Little Linguistics Goes a Long Way: Unsupervised Segmentation with Limited Language Specific Guidance
We present de-lexical segmentation, a linguistically motivated alternative to greedy or other unsupervised methods, requiring only minimal language specific input. Our technique involves creating a small grammar of close…
When Less Is More? Diagnosing ASR Predictions in Sardinian via Layer-Wise Decoding
Recent studies have shown that intermediate layers in multilingual speech models often encode more phonetically accurate representations than the final output layer. In this work, we apply a layer-wise decoding strategy …