Text-in-Context: Token-Level Error Detection for Table-to-Text Generation
We present our Charles-UPF submission for the Shared Task on Evaluating Accuracy in Generated Texts at INLG 2021. Our system can detect the errors automatically using a combination of a rule-based natural language generation (NLG) system and pretrained language models (LMs). We first utilize a rule-based NLG system to generate sentences with facts that can be derived from the input. For each sentence we evaluate, we select a subset of facts which are relevant by measuring semantic similarity to the sentence in question. Finally, we finetune a pretrained language model on annotated data along with the relevant facts for fine-grained error detection. On the test set, we achieve 69% recall and 75% precision with a model trained on a mixture of human-annotated and synthetic data.
Code (1)
Tasks
Language ModelingLanguage ModellingSemantic SimilaritySemantic Textual SimilaritySentenceTable-to-Text GenerationText GenerationSimilar Papers 제목 키워드 기반
Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models
Discrete masked diffusion language models such as LLaDA generate text through iterative denoising, where mask tokens are progressively replaced with predicted tokens. LLaDA2.1 introduced a Token-to-Token (T2T) editing me…
Instruction FollowingTowards Token-Level Text Anomaly Detection
Despite significant progress in text anomaly detection for web applications such as spam filtering and fake news detection, existing methods are fundamentally limited to document-level analysis, unable to identify which …
Fake News DetectionAnomaly DetectionImproving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selection
Nowadays, most methods in end-to-end contextual speech recognition bias the recognition process towards contextual knowledge. Since all-neural contextual biasing methods rely on phrase-level contextual modeling and atten…
speech-recognitionSpeech RecognitionMulti-Level Contextual Token Relation Modeling for Machine-Generated Text Detection
Machine-generated texts (MGTs) pose risks such as disinformation and phishing, underscoring the need for reliable detection. Metric-based methods, which extract statistically distinguishable features of MGTs, are often m…
Text DetectionZeroED: Hybrid Zero-shot Error Detection through Large Language Model Reasoning
Error detection (ED) in tabular data is crucial yet challenging due to diverse error types and the need for contextual understanding. Traditional ED methods often rely heavily on manual criteria and labels, making them l…
In-Context LearningLanguage ModelingLanguage ModellingLarge Language Model