Comparison and Adaptation of Automatic Evaluation Metrics for Quality Assessment of Re-Speaking
Re-speaking is a mechanism for obtaining high quality subtitles for use in live broadcast and other public events. Because it relies on humans performing the actual re-speaking, the task of estimating the quality of the results is non-trivial. Most organisations rely on humans to perform the actual quality assessment, but purely automatic methods have been developed for other similar problems, like Machine Translation. This paper will try to compare several of these methods: BLEU, EBLEU, NIST, METEOR, METEOR-PL, TER and RIBES. These will then be matched to the human-derived NER metric, commonly used in re-speaking.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNERTranslationSimilar Papers 제목 키워드 기반
Assessing the Quality of MT Systems for Hindi to English Translation
Evaluation plays a vital role in checking the quality of MT output. It is done either manually or automatically. Manual evaluation is very time consuming and subjective, hence use of automatic metrics is done most of the…
TranslationEASSE: Easier Automatic Sentence Simplification Evaluation
We introduce EASSE, a Python package aiming to facilitate and standardise automatic evaluation and comparison of Sentence Simplification (SS) systems. EASSE provides a single access point to a broad range of evaluation r…
SentenceEvaluation and Ranking of Machine Translated Output in Hindi Language using Precision and Recall Oriented Metrics
Evaluation plays a crucial role in development of Machine translation systems. In order to judge the quality of an existing MT system i.e. if the translated output is of human translation quality or not, various automati…
Machine TranslationTranslationAccounting for Language Effect in the Evaluation of Cross-lingual AMR Parsers
Cross-lingual Abstract Meaning Representation (AMR) parsers are currently evaluated in comparison to gold English AMRs, despite parsing a language other than English, due to the lack of multilingual AMR evaluation metric…
Abstract Meaning RepresentationSentenceA Data Set for the Analysis of Text Quality Dimensions in Summarization Evaluation
Automatic evaluation of summarization focuses on developing a metric to represent the quality of the resulting text. However, text qualityis represented in a variety of dimensions ranging from grammaticality to readabili…