The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results
In this paper, we introduce the Eval4NLP-2021shared task on explainable quality estimation. Given a source-translation pair, this shared task requires not only to provide a sentence-level score indicating the overall quality of the translation, but also to explain this score by identifying the words that negatively impact translation quality. We present the data, annotation guidelines and evaluation setup of the shared task, describe the six participating systems, and analyze the results. To the best of our knowledge, this is the first shared task on explainable NLP evaluation metrics. Datasets and results are available at https://github.com/eval4nlp/SharedTask2021.
Code (1)
Tasks
SentenceTranslationSimilar Papers 제목 키워드 기반
IST-Unbabel 2021 Submission for the Explainable Quality Estimation Shared Task
The UMD Submission to the Explainable MT Quality Estimation Shared Task: Combining Explanation Models with Sequence Labeling
CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task
We present the joint contribution of IST and Unbabel to the WMT 2022 Shared Task on Quality Estimation (QE). Our team participated on all three subtasks: (i) Sentence and Word-level Quality Prediction; (ii) Explainable Q…
AllSentencePerturbation-based QE: An Explainable, Unsupervised Word-level Quality Estimation Method for Blackbox Machine Translation
Quality Estimation (QE) is the task of predicting the quality of Machine Translation (MT) system output, without using any gold-standard translation references. State-of-the-art QE models are supervised: they require hum…
Machine TranslationTranslationWord Sense DisambiguationFrom Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimation
Machine Translation Quality Estimation (MTQE) is the task of estimating the quality of machine-translated text in real time without the need for reference translations, which is of great importance for the development of…
Deep LearningMachine TranslationSentence