Evaluating Machine Translation in a Usage Scenario
In this document we report on a user-scenario-based evaluation aiming at assessing the performance of machine translation (MT) systems in a real context of use. We describe a sequel of experiments that has been performed to estimate the usefulness of MT and to test if improvements of MT technology lead to better performance in the usage scenario. One goal is to find the best methodology for evaluating the eventual benefit of a machine translation system in an application. The evaluation is based on the QTLeap corpus, a novel multilingual language resource that was collected through a real-life support service via chat. It is composed of naturally occurring utterances produced by users while interacting with a human technician providing answers. The corpus is available in eight different languages: Basque, Bulgarian, Czech, Dutch, English, German, Portuguese and Spanish.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translation
Pretrained character-level and byte-level language models have been shown to be competitive with popular subword models across a range of Natural Language Processing (NLP) tasks. However, there has been little research o…
Machine TranslationNMTTranslationEvaluating Domain Adaptation for Machine Translation Across Scenarios
Evaluating a Machine Translation System in a Technical Support Scenario
Which Tokens Need Context? A Reference-Based Analysis of Translation Responsibility Using Fertility and Entropy
When humans translate, not every word depends equally on the surrounding context. Some tokens, particularly function words like pronouns and auxiliaries, rely heavily on preceding or following sentences, while others, su…
Machine TranslationCombining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation
Large language model (LLM) shows promising performances in a variety of downstream tasks, such as machine translation (MT). However, using LLMs for translation suffers from high computational costs and significant latenc…
Language ModelingLanguage ModellingLarge Language ModelMachine Translation+4