Evaluating Pretrained Transformer Models for Entity Linking inTask-Oriented Dialog
The wide applicability of pretrained transformer models (PTMs) for natural language tasks is well demonstrated, but their ability to comprehend short phrases of text is less explored. To this end, we evaluate different PTMs from the lens of unsupervised Entity Linking in task-oriented dialog across 5 characteristics– syntactic, semantic, short-forms, numeric and phonetic. Our results demonstrate that several of the PTMs produce sub-par results when compared to traditional techniques, albeit competitive to other neural baselines. We find that some of their shortcomings can be addressed by using PTMs fine-tuned for text-similarity tasks, which illustrate an improved ability in comprehending semantic and syntactic correspondences, as well as some improvements for short-forms, numeric and phonetic variations in entity mentions. We perform qualitative analysis to understand nuances in their predictions and discuss scope for further improvements.
Code (1)
Tasks
Entity Linkingtext similaritySimilar Papers 제목 키워드 기반
Evaluating Pretrained Transformer Models for Entity Linking in Task-Oriented Dialog
The wide applicability of pretrained transformer models (PTMs) for natural language tasks is well demonstrated, but their ability to comprehend short phrases of text is less explored. To this end, we evaluate different P…
Entity Linkingtext similarityEmpirical Evaluation of Pretraining Strategies for Supervised Entity Linking
In this work, we present an entity linking model which combines a Transformer architecture with large scale pretraining from Wikipedia links. Our model achieves the state-of-the-art on two commonly used entity linking da…
Entity LinkingEvaluating Entity Linking: An Analysis of Current Benchmark Datasets and a Roadmap for Doing a Better Job
Entity linking has become a popular task in both natural language processing and semantic web communities. However, we find that the benchmark datasets for entity linking tasks do not accurately evaluate entity linking s…
Entity LinkingLearn to Not Link: Exploring NIL Prediction in Entity Linking
Entity linking models have achieved significant success via utilizing pretrained language models to capture semantic features. However, the NIL prediction problem, which aims to identify mentions without a corresponding …
Entity LinkingPredictionLMN at SemEval-2022 Task 11: A Transformer-based System for English Named Entity Recognition
Processing complex and ambiguous named entities is a challenging research problem, but it has not received sufficient attention from the natural language processing community. In this short paper, we present our particip…
Data AugmentationEntity Linkingnamed-entity-recognitionNamed Entity Recognition+1