Eye Tracking as a Tool for Machine Translation Error Analysis
We present a preliminary study where we use eye tracking as a complement to machine translation (MT) error analysis, the task of identifying and classifying MT errors. We performed a user study where subjects read short texts translated by three MT systems and one human translation, while we gathered eye tracking data. The subjects were also asked comprehension questions about the text, and were asked to estimate the text quality. We found that there are a longer gaze time and a higher number of fixations on MT errors, than on correct parts. There are also differences in the gaze time of different error types, with word order errors having the longest gaze time. We also found correlations between eye tracking data and human estimates of text quality. Overall our study shows that eye tracking can give complementary information to error analysis, such as aiding in ranking error types for seriousness.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationReading ComprehensionTranslationSimilar Papers 제목 키워드 기반
Discriminative Language Models as a Tool for Machine Translation Error Analysis
Evaluation of Chinese-English Machine Translation of Emotion-Loaded Microblog Texts: A Human Annotated Dataset for the Quality Assessment of Emotion Translation
In this paper, we focus on how current Machine Translation (MT) tools perform on the translation of emotion-loaded texts by evaluating outputs from Google Translate according to a framework proposed in this paper. We pro…
Machine TranslationNegationTranslationTranslation Canvas: An Explainable Interface to Pinpoint and Analyze Translation Systems
With the rapid advancement of machine translation research, evaluation toolkits have become essential for benchmarking system progress. Tools like COMET and SacreBLEU offer single quality score assessments that are effec…
BenchmarkingMachine TranslationTranslationGECO-MT: The Ghent Eye-tracking Corpus of Machine Translation
In the present paper, we describe a large corpus of eye movement data, collected during natural reading of a human translation and a machine translation of a full novel. This data set, called GECO-MT (Ghent Eye tracking …
Machine TranslationTranslationLazy Low-Resource Coreference Resolution: a Study on Leveraging Black-Box Translation Tools
Large annotated corpora for coreference resolution are available for few languages. For machine translation, however, strong black-box systems exist for many languages. We empirically explore the appealing idea of levera…
coreference-resolutionCoreference ResolutionMachine TranslationTranslation