Efficient Extraction of Pseudo-Parallel Sentences from Raw Monolingual Data Using Word Embeddings
We propose a new method for extracting pseudo-parallel sentences from a pair of large monolingual corpora, without relying on any document-level information. Our method first exploits word embeddings in order to efficiently evaluate trillions of candidate sentence pairs and then a classifier to find the most reliable ones. We report significant improvements in domain adaptation for statistical machine translation when using a translation model trained on the sentence pairs extracted from in-domain monolingual corpora.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationInformation RetrievalMachine TranslationSentenceTranslationWord EmbeddingsSimilar Papers 제목 키워드 기반
Large-scale Hierarchical Alignment for Data-driven Text Rewriting
We propose a simple unsupervised method for extracting pseudo-parallel monolingual sentence pairs from comparable corpora representative of two different text styles, such as news articles and scientific papers. Our appr…
ArticlesSentenceStyle TransferText SimplificationD-Terminer: Online Demo for Monolingual and Bilingual Automatic Term Extraction
This contribution presents D-Terminer: an open access, online demo for monolingual and multilingual automatic term extraction from parallel corpora. The monolingual term extraction is based on a recurrent neural network,…
Term ExtractionTranslationDomain Adaptation of Neural Machine Translation by Lexicon Induction
It has been previously noted that neural machine translation (NMT) is very sensitive to domain shift. In this paper, we argue that this is a dual effect of the highly lexicalized nature of NMT, resulting in failure for s…
Domain AdaptationMachine TranslationNMTTranslationFiltering Back-Translated Data in Unsupervised Neural Machine Translation
Unsupervised neural machine translation (NMT) utilizes only monolingual data for training. The quality of back-translated data plays an important role in the performance of NMT systems. In back-translation, all generated…
Domain AdaptationMachine TranslationNMTSentence+1Unsupervised Extraction of Partial Translations for Neural Machine Translation
In neural machine translation (NMT), monolingual data are usually exploited through a so-called back-translation: sentences in the target language are translated into the source language to synthesize new parallel data. …
Machine TranslationNMTTranslation