Understanding the Impact of UGC Specificities on Translation Quality
This work takes a critical look at the evaluation of user-generated content automatic translation, the well-known specificities of which raise many challenges for MT. Our analyses show that measuring the average-case performance using a standard metric on a UGC test set falls far short of giving a reliable image of the UGC translation quality. That is why we introduce a new data set for the evaluation of UGC translation in which UGC specificities have been manually annotated using a fine-grained typology. Using this data set, we conduct several experiments to measure the impact of different kinds of UGC specificities on translation quality, more precisely than previously possible.
Code (0)
등록된 구현이 없습니다.
Tasks
TranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Extracting Directional and Comparable Corpora from a Multilingual Corpus for Translation Studies
Translation studies rely more and more on corpus data to examine specificities of translated texts, that can be translated from different original languages and compared to original texts. In parallel, more and more mult…
Machine TranslationTranslationComparison between NMT and PBSMT Performance for Translating Noisy User-Generated Content
This work compares the performances achieved by Phrase-Based Statistical Machine Translation systems (PB-SMT) and attention-based Neuronal Machine Translation systems (NMT) when translating User Generated Content (UGC), …
Machine TranslationNMTTranslationAre We Solving a Well-Defined Problem? A Task-Centric Perspective on Recommendation Tasks
Recommender systems (RecSys) leverage user interaction history to predict and suggest relevant items, shaping user experiences across various domains. While many studies adopt a general problem definition, i.e., to recom…
NavigateRecommendation SystemsSpecificityBERTology for Machine Translation: What BERT Knows about Linguistic Difficulties for Translation
Pre-trained transformer-based models, such as BERT, have shown excellent performance in most natural language processing benchmark tests, but we still lack a good understanding of the linguistic knowledge of BERT in Neur…
Machine TranslationNMTTranslationGuiding In-Context Learning of LLMs through Quality Estimation for Machine Translation
The quality of output from large language models (LLMs), particularly in machine translation (MT), is closely tied to the quality of in-context examples (ICEs) provided along with the query, i.e., the text to translate. …
In-Context LearningLanguage ModelingLanguage ModellingMachine Translation+1