deepQuest-py: Large and Distilled Models for Quality Estimation
We introduce deepQuest-py, a framework for training and evaluation of large and light-weight models for Quality Estimation (QE). deepQuest-py provides access to (1) state-of-the-art models based on pre-trained Transformers for sentence-level and word-level QE; (2) light-weight and efficient sentence-level models implemented via knowledge distillation; and (3) a web interface for testing models and visualising their predictions. deepQuest-py is available at https://github.com/sheffieldnlp/deepQuest-py under a CC BY-NC-SA licence.
Code (1)
Tasks
Knowledge DistillationSentenceSimilar Papers 제목 키워드 기반
DEEPQUESTION: Systematic Generation of Real-World Challenges for Evaluating LLMs Performance
LLMs often excel on standard benchmarks but falter on real-world tasks. We introduce DeepQuestion, a scalable automated framework that augments existing datasets based on Bloom's taxonomy and creates novel questions that…
deepQuest: A Framework for Neural-based Quality Estimation
Predicting Machine Translation (MT) quality can help in many practical tasks such as MT post-editing. The performance of Quality Estimation (QE) methods has drastically improved recently with the introduction of neural a…
Feature EngineeringMachine TranslationSentenceTranslationKnowledge Distillation for Quality Estimation
Quality Estimation (QE) is the task of automatically predicting Machine Translation quality in the absence of reference translations, making it applicable in real-time settings, such as translating online social media co…
Data AugmentationKnowledge DistillationMachine TranslationTranslationKnowledge distillation for fast and accurate DNA sequence correction
Accurate genome sequencing can improve our understanding of biology and the genetic basis of disease. The standard approach for generating DNA sequences from PacBio instruments relies on HMM-based models. Here, we introd…
Knowledge DistillationEnhancing AI Face Realism: Cost-Efficient Quality Improvement in Distilled Diffusion Models with a Fully Synthetic Dataset
This study presents a novel approach to enhance the cost-to-quality ratio of image generation with diffusion models. We hypothesize that differences between distilled (e.g. FLUX.1-schnell) and baseline (e.g. FLUX.1-dev) …
Image GenerationImage-to-Image Translation