Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding

2023-11-14 · Guangyu Yang, Jinghong Chen, Weizhe Lin, Bill Byrne

Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Reinforcement Learning technique, Direct Preference Optimization (DPO), can fine-tune MLLMs to get the gains of MBR without any additional computation in inference. Our method uses only a small monolingual fine-tuning set and yields significantly improved performance on multiple NMT test sets compared to MLLMs without DPO.

📄 PDF Abstract BibTeX arXiv:2311.08380

Code (1)

bruceyg/dpo-mbr 공식 구현 pytorch

Tasks

Machine TranslationNMTreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Translation

Methods 이 논문이 사용한 방법론

DPO 설명 없음

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

BASE 설명 없음

Choose the Final Translation from NMT and LLM hypotheses Using MBR Decoding: HW-TSC's Submission to the WMT24 General MT Shared Task

2024-09-23 · Zhanglin Wu, Daimeng Wei, Zongyao Li, Hengchao Shang 외

This paper presents the submission of Huawei Translate Services Center (HW-TSC) to the WMT24 general machine translation (MT) shared task, where we participate in the English to Chinese (en2zh) language pair. Similar to …

Ensemble Learning Language Modeling Language Modelling Large Language Model+3

Cross-lingual Human-Preference Alignment for Neural Machine Translation with Direct Quality Optimization

2024-09-26 · Kaden Uhlig, Joern Wuebker, Raphael Reinauer, John DeNero

Reinforcement Learning from Human Feedback (RLHF) and derivative techniques like Direct Preference Optimization (DPO) are task-alignment algorithms used to repurpose general, foundational models for specific tasks. We sh…

Machine Translation NMT Translation

Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation

2026-04-28 · Mehrdad Ghassabi, Spehr Rajabi, Hamidreza Baradaran Kashani, Sadra Hakim 외 arxiv

Contemporary neural machine translation (NMT) systems are almost exclusively built by training on supervised parallel data. Despite the tremendous progress achieved, these systems still exhibit persistent translation err…

Reinforcement Learning Machine Translation

CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation

2025-01-23 · Guofeng Cui, Pichao Wang, Yang Liu, Zemian Ke 외

Large language models (LLMs) have shown great potential in natural language processing tasks, but their application to machine translation (MT) remains challenging due to pretraining on English-centric data and the compl…

Decoder Machine Translation Sentence Translation

Word Alignment as Preference for Machine Translation

2024-05-15 · Qiyu Wu, Masaaki Nagata, Zhongtao Miao, Yoshimasa Tsuruoka

The problem of hallucination and omission, a long-standing problem in machine translation (MT), is more pronounced when a large language model (LLM) is used in MT because an LLM itself is susceptible to these phenomena. …

Hallucination Language Modelling Large Language Model Machine Translation+2