Pairwise Neural Machine Translation Evaluation
We present a novel framework for machine translation evaluation using neural networks in a pairwise setting, where the goal is to select the better translation from a pair of hypotheses, given the reference translation. In this framework, lexical, syntactic and semantic information from the reference and the two hypotheses is compacted into relatively small distributed vector representations, and fed into a multi-layer neural network that models the interaction between each of the hypotheses and the reference, as well as between the two hypotheses. These compact representations are in turn based on word and sentence embeddings, which are learned using neural networks. The framework is flexible, allows for efficient learning and classification, and yields correlation with humans that rivals the state of the art.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSentenceSentence EmbeddingsTranslationSimilar Papers 제목 키워드 기반
PEAR: Pairwise Evaluation for Automatic Relative Scoring in Machine Translation
We present PEAR (Pairwise Evaluation for Automatic Relative Scoring), a supervised quality estimation (QE) metric family that reframes reference-free machine translation (MT) evaluation as a graded pairwise comparison. G…
Machine TranslationMachine Translation Evaluation Meets Community Question Answering
We explore the applicability of machine translation evaluation (MTE) methods to a very different problem: answer ranking in community Question Answering. In particular, we adopt a pairwise neural network (NN) architectur…
Community Question AnsweringMachine TranslationQuestion AnsweringTranslationKnockout LLM Assessment: Using Large Language Models for Evaluations through Iterative Pairwise Comparisons
Large Language Models (LLMs) have shown to be effective evaluators across various domains such as machine translations or the scientific domain. Current LLM-as-a-Judge approaches rely mostly on individual assessments or …
Machine TranslationHas Machine Translation Achieved Human Parity? A Case for Document-level Evaluation
Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese--English news translation task. We empirically test this claim with alternative evaluation p…
Machine TranslationSentenceTranslationSelf-Supervised Learning for Pairwise Data Refinement
Pairwise data automatically constructed from weakly supervised signals has been widely used for training deep learning models. Pairwise datasets such as parallel texts can have uneven quality levels overall, but usually …
Machine TranslationSelf-Supervised LearningTranslation