Exploiting Sentence Order in Document Alignment
We present a simple document alignment method that incorporates sentence order information in both candidate generation and candidate re-scoring. Our method results in 61% relative reduction in error compared to the best previously published result on the WMT16 document alignment shared task. Our method improves downstream MT performance on web-scraped Sinhala--English documents from ParaCrawl, outperforming the document alignment method used in the most recent ParaCrawl release. It also outperforms a comparable corpora method which uses the same multilingual embeddings, demonstrating that exploiting sentence order is beneficial even if the end goal is sentence-level bitext.
Code (1)
Tasks
SentenceSimilar Papers 제목 키워드 기반
Exploiting Parallel Corpora to Improve Multilingual Embedding based Document and Sentence Alignment
Multilingual sentence representations pose a great advantage for low-resource languages that do not have enough data to build monolingual models on their own. These multilingual sentence representations have been separat…
SentenceNon-Autoregressive Document-Level Machine Translation
Non-autoregressive translation (NAT) models achieve comparable performance and superior speed compared to auto-regressive translation (AT) models in the context of sentence-level machine translation (MT). However, their …
Document Level Machine TranslationMachine TranslationSentenceText Generation+1OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment
Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to words and subwords. Existing tools, however, typically specialize in a …
Long-Context UnderstandingSelf-Supervised LearningWord AlignmentSentAlign: Accurate and Scalable Sentence Alignment
We present SentAlign, an accurate sentence alignment tool designed to handle very large parallel document pairs. Given user-defined parameters, the alignment algorithm evaluates all possible alignment paths in fairly lar…
Machine TranslationSentenceTranslationA Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection
An essential task of most Question Answering (QA) systems is to re-rank the set of answer candidates, i.e., Answer Sentence Selection (A2S). These candidates are typically sentences either extracted from one or more docu…
Question AnsweringSentence