Addressing Exposure Bias With Document Minimum Risk Training: Cambridge at the WMT20 Biomedical Translation Task
The 2020 WMT Biomedical translation task evaluated Medline abstract translations. This is a small-domain translation task, meaning limited relevant training data with very distinct style and vocabulary. Models trained on such data are susceptible to exposure bias effects, particularly when training sentence pairs are imperfect translations of each other. This can result in poor behaviour during inference if the model learns to neglect the source sentence. The UNICAM entry addresses this problem during fine-tuning using a robust variant on Minimum Risk Training. We contrast this approach with data-filtering to remove `problem' training examples. Under MRT fine-tuning we obtain good results for both directions of English-German and English-Spanish biomedical translation. In particular we achieve the best English-to-Spanish translation result and second-best Spanish-to-English result, despite using only single models with no ensembling.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceTranslationSimilar Papers 제목 키워드 기반
On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation
The standard training algorithm in neural machine translation (NMT) suffers from exposure bias, and alternative algorithms have been proposed to mitigate this. However, the practical impact of exposure bias is under deba…
HallucinationMachine TranslationNMTTranslationCOMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training
Optical chemical structure recognition (OCSR) translates molecular images into machine-readable representations like SMILES strings or molecular graphs, but remains challenging in real-world documents due to inexhaustibl…
Choosing Online Experiment Designs under Interference in Ads, Recommendations, and Member-Experience Systems
Online experiments in ads, recommendation, and member-experience systems are often planned before the dominant interference mechanism is known. A treatment may propagate through budgets, inventory, producer exposure, gra…
Options on CPPI with guaranteed minimum equity exposure
In the present paper we provide a two-step principal protection strategy obtained by combining a modification of the Constant Proportion Portfolio Insurance (CPPI) algorithm and a classical Option Based Portfolio Insuran…
Mitigating Exposure Bias in Risk-Aware Time Series Forecasting with Soft Tokens
Autoregressive forecasting is central to predictive control in diabetes and hemodynamic management, where different operating zones carry different clinical risks. Standard models trained with teacher forcing suffer from…
Time Series ForecastingTrajectory Forecasting