VERTa: Facing a Multilingual Experience of a Linguistically-based MT Evaluation
There are several MT metrics used to evaluate translation into Spanish, although most of them use partial or little linguistic information. In this paper we present the multilingual capability of VERTa, an automatic MT metric that combines linguistic information at lexical, morphological, syntactic and semantic level. In the experiments conducted we aim at identifying those linguistic features that prove the most effective to evaluate adequacy in Spanish segments. This linguistic information is tested both as independent modules (to observe what each type of feature provides) and in a combinatory fastion (where different kinds of information interact with each other). This allows us to extract the optimal combination. In addition we compare these linguistic features to those used in previous versions of VERTa aimed at evaluating adequacy for English segments. Finally, experiments show that VERTa can be easily adapted to other languages than English and that its collaborative approach correlates better with human judgements on adequacy than other well-known metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
Sentiment AnalysisTranslationSimilar Papers 제목 키워드 기반
VERTa: a Linguistically-motivated Metric at the WMT15 Metrics Task
IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages
As large language models (LLMs) see increasing adoption across the globe, it is imperative for LLMs to be representative of the linguistic diversity of the world. India is a linguistically diverse country of 1.4 Billion …
Cross-Lingual Question AnsweringDiversityMachine TranslationQuestion AnsweringUniversal Dependencies v1: A Multilingual Treebank Collection
Cross-linguistically consistent annotation is necessary for sound comparative evaluation and cross-lingual learning experiments. It is also useful for multilingual system development and comparative linguistic studies. U…
Average Is Not Enough: Caveats of Multilingual Evaluation
This position paper discusses the problem of multilingual evaluation. Using simple statistics, such as average language performance, might inject linguistic biases in favor of dominant language families into evaluation m…
PositionAutonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning
Professional race-car drivers can execute extreme overtaking maneuvers. However, existing algorithms for autonomous overtaking either rely on simplified assumptions about the vehicle dynamics or try to solve expensive tr…
Car Racingreinforcement-learningReinforcement LearningReinforcement Learning (RL)