Learning to Stop in Structured Prediction for Neural Machine Translation
Beam search optimization resolves many issues in neural machine translation. However, this method lacks principled stopping criteria and does not learn how to stop during training, and the model naturally prefers the longer hypotheses during the testing time in practice since they use the raw score instead of the probability-based score. We propose a novel ranking method which enables an optimal beam search stopping criteria. We further introduce a structured prediction loss function which penalizes suboptimal finished candidates produced by beam search during training. Experiments of neural machine translation on both synthetic data and real languages (German-to-English and Chinese-to-English) demonstrate our proposed methods lead to better length and BLEU score.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationStructured PredictionTranslationSimilar Papers 제목 키워드 기반
Bandit Structured Prediction for Learning from Partial Feedback in Statistical Machine Translation
We present an approach to structured prediction from bandit feedback, called Bandit Structured Prediction, where only the value of a task loss function at a single predicted point, instead of a correct structure, is obse…
Machine TranslationRerankingStructured PredictionTranslationFuture-Prediction-Based Model for Neural Machine Translation
We propose a novel model for Neural Machine Translation (NMT). Different from the conventional method, our model can predict the future text length and words at each decoding time step so that the generation can be helpe…
Future predictionMachine TranslationNMTPrediction+1A General Framework for Adaptation of Neural Machine Translation to Simultaneous Translation
Despite the success of neural machine translation (NMT), simultaneous neural machine translation (SNMT), the task of translating in real time before a full sentence has been observed, remains challenging due to the synta…
Machine TranslationNMTSentenceTranslationBreaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation
Beam search is widely used in neural machine translation, and usually improves translation quality compared to greedy search. It has been widely observed that, however, beam sizes larger than 5 hurt translation quality. …
Machine TranslationTranslation