deepQuest: A Framework for Neural-based Quality Estimation
Predicting Machine Translation (MT) quality can help in many practical tasks such as MT post-editing. The performance of Quality Estimation (QE) methods has drastically improved recently with the introduction of neural approaches to the problem. However, thus far neural approaches have only been designed for word and sentence-level prediction. We present a neural framework that is able to accommodate neural QE approaches at these fine-grained levels and generalize them to the level of documents. We test the framework with two sentence-level neural QE approaches: a state of the art approach that requires extensive pre-training, and a new light-weight approach that we propose, which employs basic encoders. Our approach is significantly faster and yields performance improvements for a range of document-level quality estimation tasks. To our knowledge, this is the first neural architecture for document-level QE. In addition, for the first time we apply QE models to the output of both statistical and neural MT systems for a series of European languages and highlight the new challenges resulting from the use of neural MT.
Code (2)
Tasks
Feature EngineeringMachine TranslationSentenceTranslationSimilar Papers 제목 키워드 기반
deepQuest-py: Large and Distilled Models for Quality Estimation
We introduce deepQuest-py, a framework for training and evaluation of large and light-weight models for Quality Estimation (QE). deepQuest-py provides access to (1) state-of-the-art models based on pre-trained Transforme…
Knowledge DistillationSentenceDEEPQUESTION: Systematic Generation of Real-World Challenges for Evaluating LLMs Performance
LLMs often excel on standard benchmarks but falter on real-world tasks. We introduce DeepQuestion, a scalable automated framework that augments existing datasets based on Bloom's taxonomy and creates novel questions that…
OpenKiwi: An Open Source Framework for Quality Estimation
We introduce OpenKiwi, a PyTorch-based open source framework for translation quality estimation. OpenKiwi supports training and testing of word-level and sentence-level quality estimation systems, implementing the winnin…
NMTSentenceTranslationData Quality Estimation Framework for Faster Tax Code Classification
This paper describes a novel framework to estimate the data quality of a collection of product descriptions to identify required relevant information for accurate product listing classification for tax-code assignment. O…
AttributeAttribute Value ExtractionClassificationCode Classification+1Modeling the Uncertainty with Maximum Discrepant Students for Semi-supervised 2D Pose Estimation
Semi-supervised pose estimation is a practically challenging task for computer vision. Although numerous excellent semi-supervised classification methods have emerged, these methods typically use confidence to evaluate t…
2D Pose EstimationPose Estimation