Evaluating the Translation Performance of Large Language Models Based on Euas-20
In recent years, with the rapid development of deep learning technology, large language models (LLMs) such as BERT and GPT have achieved breakthrough results in natural language processing tasks. Machine translation (MT), as one of the core tasks of natural language processing, has also benefited from the development of large language models and achieved a qualitative leap. Despite the significant progress in translation performance achieved by large language models, machine translation still faces many challenges. Therefore, in this paper, we construct the dataset Euas-20 to evaluate the performance of large language models on translation tasks, the translation ability on different languages, and the effect of pre-training data on the translation ability of LLMs for researchers and developers.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Diffusion-based Surrogate Model for Time-varying Underwater Acoustic Channels
Accurate modeling of time-varying underwater acoustic channels is essential for the design, evaluation, and deployment of reliable underwater communication systems. Conventional physics models require detailed environmen…
Towards Large Language Model driven Reference-less Translation Evaluation for English and Indian Languages
With the primary focus on evaluating the effectiveness of large language models for automatic reference-less translation assessment, this work presents our experiments on mimicking human direct assessment to evaluate the…
Language ModelingLanguage ModellingLarge Language ModelTranslation+1DATScore: Evaluating Translation with Data Augmented Translations
The rapid development of large pretrained language models has revolutionized not only the field of Natural Language Generation (NLG) but also its evaluation. Inspired by the recent work of BARTScore: a metric leveraging …
Data AugmentationLanguage ModelingLanguage ModellingMachine Translation+2Asm2SrcEval: Evaluating Large Language Models for Assembly-to-Source Code Translation
Assembly-to-source code translation is a critical task in reverse engineering, cybersecurity, and software maintenance, yet systematic benchmarks for evaluating large language models on this problem remain scarce. In thi…
Code TranslationProverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model
Despite achieving remarkable performance, machine translation (MT) research remains underexplored in terms of translating cultural elements in languages, such as idioms, proverbs, and colloquial expressions. This paper i…
Language ModelingLanguage ModellingLarge Language ModelMachine Translation+2