Do GPTs Produce Less Literal Translations?
Large Language Models (LLMs) such as GPT-3 have emerged as general-purpose language models capable of addressing many natural language generation or understanding tasks. On the task of Machine Translation (MT), multiple works have investigated few-shot prompting mechanisms to elicit better translations from LLMs. However, there has been relatively little investigation on how such translations differ qualitatively from the translations generated by standard Neural Machine Translation (NMT) models. In this work, we investigate these differences in terms of the literalness of translations produced by the two systems. Using literalness measures involving word alignment and monotonicity, we find that translations out of English (E-X) from GPTs tend to be less literal, while exhibiting similar or better scores on MT quality metrics. We demonstrate that this finding is borne out in human evaluations as well. We then show that these differences are especially pronounced when translating sentences that contain idiomatic expressions.
Code (1)
Tasks
Machine TranslationNMTText GenerationTranslationWord AlignmentMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Testing the Deliteralization Hypothesis in Human and Machine Translation
The recent shift from dedicated NMT systems to general-purpose LLMs has reshaped machine translation, with LLMs reported to produce more fluent, less literal output than their predecessors. We test whether this shift ext…
Machine TranslationDetecting Non-literal Translations by Fine-tuning Cross-lingual Pre-trained Language Models
Human-generated non-literal translations reflect the richness of human languages and are sometimes indispensable to ensure adequacy and fluency. Non-literal translations are difficult to produce even for human translator…
Language ModellingSentenceTranslationThe Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities
Fine-tuning large language models (LLMs) for machine translation has shown improvements in overall translation quality. However, it is unclear what is the impact of fine-tuning on desirable LLM behaviors that are not pre…
Machine TranslationTranslationComparing Formulaic Language in Human and Machine Translation: Insight from a Parliamentary Corpus
A recent study has shown that, compared to human translations, neural machine translations contain more strongly-associated formulaic sequences made of relatively high-frequency words, but far less strongly-associated fo…
ArticlesMachine TranslationTranslationHow Good Are LLMs for Literary Translation, Really? Literary Translation Evaluation with Humans and LLMs
Recent research has focused on literary machine translation (MT) as a new challenge in MT. However, the evaluation of literary MT remains an open problem. We contribute to this ongoing discussion by introducing LITEVAL-C…
2kMachine TranslationTranslation