MIPE: A Metric Independent Pipeline for Effective Code-Mixed NLG Evaluation
Code-mixing is a phenomenon of mixing words and phrases from two or more languages in a single utterance of speech and text. Due to the high linguistic diversity, code-mixing presents several challenges in evaluating standard natural language generation (NLG) tasks. Various widely popular metrics perform poorly with the code-mixed NLG tasks. To address this challenge, we present a metric independent evaluation pipeline MIPE that significantly improves the correlation between evaluation metrics and human judgments on the generated code-mixed text. As a use case, we demonstrate the performance of MIPE on the machine-generated Hinglish (code-mixing of Hindi and English languages) sentences from the HinGE corpus. We can extend the proposed evaluation strategy to other code-mixed language pairs, NLG tasks, and evaluation metrics with minimal to no effort.
Code (0)
등록된 구현이 없습니다.
Tasks
Diversitynlg evaluationText GenerationSimilar Papers 제목 키워드 기반
Improving Paratope and Epitope Prediction by Multi-Modal Contrastive Learning and Interaction Informativeness Estimation
Accurately predicting antibody-antigen binding residues, i.e., paratopes and epitopes, is crucial in antibody design. However, existing methods solely focus on uni-modal data (either sequence or structure), disregarding …
Antibody-antigen binding predictionContrastive LearningInformativenessMultiple Invertible and Partial-Equivariant Function for Latent Vector Transformation to Enhance Disentanglement in VAEs
Disentanglement learning is a core issue for understanding and re-using trained information in Variational AutoEncoder (VAE), and effective inductive bias has been reported as a key factor. However, the actual implementa…
DisentanglementInductive BiasLearning Descriptive Image Captioning via Semipermeable Maximum Likelihood Estimation
Image captioning aims to describe visual content in natural language. As 'a picture is worth a thousand words', there could be various correct descriptions for an image. However, with maximum likelihood estimation as the…
Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning
In this work, we explore the way to perform named entity recognition (NER) using only unlabeled data and named entity dictionaries. To this end, we formulate the task as a positive-unlabeled (PU) learning problem and acc…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1Detection of Metallo-\b{eta}-Lactamases-Encoding Genes Among Clinical Isolates of Escherichia Coli in a Tertiary Care Hospital, Malaysia
The multidrug resistant Escherichia coli strains causes multiple clinical infections and has become a rising problem globally. The metallo-\b{eta}-lactamases encoding genes are very severe in gram-negative bacteria espec…
Management