Guiding Large Language Models to Post-Edit Machine Translation with Error Annotations
Machine Translation (MT) remains one of the last NLP tasks where large language models (LLMs) have not yet replaced dedicated supervised systems. This work exploits the complementary strengths of LLMs and supervised MT by guiding LLMs to automatically post-edit MT with external feedback on its quality, derived from Multidimensional Quality Metric (MQM) annotations. Working with LLaMA-2 models, we consider prompting strategies varying the nature of feedback provided and then fine-tune the LLM to improve its ability to exploit the provided guidance. Through experiments on Chinese-English, English-German, and English-Russian MQM data, we demonstrate that prompting LLMs to post-edit MT improves TER, BLEU and COMET scores, although the benefits of fine-grained feedback are not clear. Fine-tuning helps integrate fine-grained feedback more effectively and further improves translation quality based on both automatic and human evaluation.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Leveraging GPT-4 for Automatic Translation Post-Editing
While Neural Machine Translation (NMT) represents the leading approach to Machine Translation (MT), the outputs of NMT models still require translation post-editing to rectify errors and enhance quality under critical se…
Machine TranslationNMTTranslationThe Post-Editing Workflow: Training Challenges for LSPs, Post-Editors and Academia
Language technology is already largely adopted by most Language Service Providers (LSPs) and integrated into their traditional translation processes. In this context, there are many different approaches to applying Post-…
Machine TranslationManagementTranslationPost-edits Are Preferences Too
Preference Optimization (PO) techniques are currently one of the state of the art techniques for fine-tuning large language models (LLMs) on pairwise preference feedback from human annotators. However, in machine transla…
Machine TranslationTranslationExtending CREAMT: Leveraging Large Language Models for Literary Translation Post-Editing
Post-editing machine translation (MT) for creative texts, such as literature, requires balancing efficiency with the preservation of creativity and style. While neural MT systems struggle with these challenges, large lan…
Machine TranslationTranslationStudying Post-Editese in a Professional Context: A Pilot Study
The past few years have seen the multiplication of studies on post-editese, following the massive adoption of post-editing in professional translation workflows. These studies mainly rely on the comparison of post-edited…
Machine TranslationTranslation