VERTa: a Linguistically-motivated Metric at the WMT15 Metrics Task

VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation

There are several MT metrics used to evaluate translation into Spanish, although most of them use partial or little linguistic information. In this paper we present the multilingual capability of VERTa, an automatic MT m…

VERTa participation in the WMT14 Metrics Task

LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translation

Existing MT evaluation frameworks, including automatic metrics and human evaluation schemes such as Multidimensional Quality Metrics (MQM), are largely language-agnostic. However, they often fail to capture dialect- and …

VERTa: Linguistic features in MT evaluation

In the last decades, a wide range of automatic metrics that use linguistic knowledge has been developed. Some of them are based on lexical information, such as METEOR; others rely on the use of syntax, either using const…

BackTranslation2.0 -- A Linguistically Motivated Metric to Assess Sign Language Production

Sign Languages (SLs) are the primary means of communication for millions of deaf individuals, yet existing evaluation metrics for generated SL remain simplistic and poorly aligned with human judgements. We introduce Back…