Machine Translation Pre-training for Data-to-Text Generation - A Case Study in Czech
While there is a large body of research studying deep learning methods for text generation from structured data, almost all of it focuses purely on English. In this paper, we study the effectiveness of machine translation based pre-training for data-to-text generation in non-English languages. Since the structured data is generally expressed in English, text generation into other languages involves elements of translation, transliteration and copying - elements already encoded in neural machine translation systems. Moreover, since data-to-text corpora are typically small, this task can benefit greatly from pre-training. We conduct experiments on Czech, a morphologically complex language. Results show that machine translation pre-training lets us train endto-end models that significantly improve upon unsupervised pre-training and linguistically informed pipelined neural systems, as judged by automatic metrics and human evaluation. We also show that this approach enjoys several desirable properties, including improved performance in low data scenarios and applicability to low resource languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Data-to-Text GenerationMachine TranslationText GenerationTranslationTransliterationUnsupervised Pre-trainingSimilar Papers 제목 키워드 기반
Machine Translation Pre-training for Data-to-Text Generation -- A Case Study in Czech
While there is a large body of research studying deep learning methods for text generation from structured data, almost all of it focuses purely on English. In this paper, we study the effectiveness of machine translatio…
Data-to-Text GenerationMachine TranslationText GenerationTranslation+1A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond
Non-autoregressive (NAR) generation, which is first proposed in neural machine translation (NMT) to speed up inference, has attracted much attention in both machine learning and natural language processing communities. W…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Dialogue GenerationGrammatical Error Correction+11Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation
Subword segmenters like BPE operate as a preprocessing step in neural machine translation and other (conditional) language models. They are applied to datasets before training, so translation or text generation quality r…
Machine TranslationSentenceText GenerationTranslationXDLM: Cross-lingual Diffusion Language Model for Machine Translation
Recently, diffusion models have excelled in image generation tasks and have also been applied to neural language processing (NLP) for controllable text generation. However, the application of diffusion models in a cross-…
Image GenerationLanguage ModelingLanguage ModellingMachine Translation+2Machine Translation Aided Bilingual Data-to-Text Generation and Semantic Parsing
We present a system for bilingual Data-ToText Generation and Semantic Parsing. We use a text-to-text generator to learn a single model that works for both languages on each of the tasks. The model is aided by machine tra…
Data-to-Text GenerationMachine TranslationSemantic ParsingText Generation+1