paper-with-me

Papers

An Exploratory Study on Multilingual Quality Estimation

2020-12-01 · Asian Chapter of the Association for Computational Linguistics 2020 · Shuo Sun, Marina Fomicheva, Fr{\'e}d{\'e}ric Blain, Vishrav Chaudhary, Ahmed El-Kishky, Adithya Renduchintala, Francisco Guzm{\'a}n, Lucia Specia

Predicting the quality of machine translation has traditionally been addressed with language-specific models, under the assumption that the quality label distribution or linguistic features exhibit traits that are not shared across languages. An obvious disadvantage of this approach is the need for labelled data for each given language pair. We challenge this assumption by exploring different approaches to multilingual Quality Estimation (QE), including using scores from translation models. We show that these outperform single-language models, particularly in less balanced quality label distributions and low-resource settings. In the extreme case of zero-shot QE, we show that it is possible to accurately predict quality for any given new language from models trained on other languages. Our findings indicate that state-of-the-art neural QE models based on powerful pre-trained representations generalise well across languages, making them more applicable in real-world settings.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Machine TranslationTranslation

Similar Papers 제목 키워드 기반

Together We Can: Multilingual Automatic Post-Editing for Low-Resource Languages

2024-10-23 · Sourabh Deoghare, Diptesh Kanojia, Pushpak Bhattacharyya

This exploratory study investigates the potential of multilingual Automatic Post-Editing (APE) systems to enhance the quality of machine translations for low-resource Indo-Aryan languages. Focusing on two closely related…

Automatic Post-EditingData AugmentationDomain AdaptationMulti-Task Learning

An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers

2021-05-31 · ACL 2021 5 · Tharindu Ranasinghe, Constantin Orasan, Ruslan Mitkov

Most studies on word-level Quality Estimation (QE) of machine translation focus on language-specific models. The obvious disadvantages of these approaches are the need for labelled data for each language pair and the hig…

Machine TranslationTranslation

A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models

2024-02-21 · Boyang Xue, Hongru Wang, Rui Wang, Sheng Wang 외

The tendency of Large Language Models (LLMs) to generate hallucinations raises concerns regarding their reliability. Therefore, confidence estimations indicating the extent of trustworthiness of the generations become es…

MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models

2024-10-16 · Boyang Xue, Hongru Wang, Rui Wang, Sheng Wang 외

The tendency of Large Language Models (LLMs) to generate hallucinations raises concerns regarding their reliability. Therefore, confidence estimations indicating the extent of trustworthiness of the generations become es…

Papago’s Submission for the WMT21 Quality Estimation Shared Task

2021-11-01 · WMT (EMNLP) 2021 11 · Seunghyun Lim, Hantae Kim, Hyunjoong Kim

This paper describes Papago submission to the WMT 2021 Quality Estimation Task 1: Sentence-level Direct Assessment. Our multilingual Quality Estimation system explores the combination of Pretrained Language Models and Mu…

Knowledge DistillationMulti-Task LearningSentence