An Empirical Study of Generation Order for Machine Translation
In this work, we present an empirical study of generation order for machine translation. Building on recent advances in insertion-based modeling, we first introduce a soft order-reward framework that enables us to train models to follow arbitrary oracle generation policies. We then make use of this framework to explore a large variety of generation orders, including uninformed orders, location-based orders, frequency-based orders, content-based orders, and model-based orders. Curiously, we find that for the WMT'14 English $\to$ German translation task, order does not have a substantial impact on output quality, with unintuitive orderings such as alphabetical and shortest-first matching the performance of a standard Transformer. This demonstrates that traditional left-to-right generation is not strictly necessary to achieve high performance. On the other hand, results on the WMT'18 English $\to$ Chinese task tend to vary more widely, suggesting that translation for less well-aligned language pairs may be more sensitive to generation order.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Data Ordering Patterns for Neural Machine Translation: An Empirical Study
Recent works show that ordering of the training data affects the model performance for Neural Machine Translation. Several approaches involving dynamic data ordering and data sharding based on curriculum learning have be…
Machine TranslationTranslationHow to Design Translation Prompts for ChatGPT: An Empirical Study
The recently released ChatGPT has demonstrated surprising abilities in natural language understanding and natural language generation. Machine translation relies heavily on the abilities of language understanding and gen…
Machine TranslationNatural Language UnderstandingText GenerationTranslationExplicit Reordering for Neural Machine Translation
In Transformer-based neural machine translation (NMT), the positional encoding mechanism helps the self-attention networks to learn the source representation with order dependency, which makes the Transformer-based NMT a…
Machine TranslationNMTSentenceTranslationPhrase Based Language Model for Statistical Machine Translation: Empirical Study
Reordering is a challenge to machine translation (MT) systems. In MT, the widely used approach is to apply word based language model (LM) which considers the constituent units of a sentence as words. In speech recognitio…
Language ModelingLanguage ModellingMachine TranslationRe-Ranking+4Beyond General Purpose Machine Translation: The Need for Context-specific Empirical Research to Design for Appropriate User Trust
Machine Translation (MT) has the potential to help people overcome language barriers and is widely used in high-stakes scenarios, such as in hospitals. However, in order to use MT reliably and safely, users need to under…
Machine TranslationTranslation