Evaluating Pretrained Transformer Models for Entity Linking in Task-Oriented Dialog
The wide applicability of pretrained transformer models (PTMs) for natural language tasks is well demonstrated, but their ability to comprehend short phrases of text is less explored. To this end, we evaluate different PTMs from the lens of unsupervised Entity Linking in task-oriented dialog across 5 characteristics -- syntactic, semantic, short-forms, numeric and phonetic. Our results demonstrate that several of the PTMs produce sub-par results when compared to traditional techniques, albeit competitive to other neural baselines. We find that some of their shortcomings can be addressed by using PTMs fine-tuned for text-similarity tasks, which illustrate an improved ability in comprehending semantic and syntactic correspondences, as well as some improvements for short-forms, numeric and phonetic variations in entity mentions. We perform qualitative analysis to understand nuances in their predictions and discuss scope for further improvements. Code can be found at https://github.com/murali1996/el_tod
Code (1)
Tasks
Entity Linkingtext similaritySimilar Papers 제목 키워드 기반
Evaluating Pretrained Transformer Models for Entity Linking inTask-Oriented Dialog
The wide applicability of pretrained transformer models (PTMs) for natural language tasks is well demonstrated, but their ability to comprehend short phrases of text is less explored. To this end, we evaluate different P…
Entity Linkingtext similarityEvaluating Entity Linking: An Analysis of Current Benchmark Datasets and a Roadmap for Doing a Better Job
Entity linking has become a popular task in both natural language processing and semantic web communities. However, we find that the benchmark datasets for entity linking tasks do not accurately evaluate entity linking s…
Entity LinkingLMN at SemEval-2022 Task 11: A Transformer-based System for English Named Entity Recognition
Processing complex and ambiguous named entities is a challenging research problem, but it has not received sufficient attention from the natural language processing community. In this short paper, we present our particip…
Data AugmentationEntity Linkingnamed-entity-recognitionNamed Entity Recognition+1LMN at SemEval-2022 Task 11: A Transformer-based System for English Named Entity Recognition
Processing complex and ambiguous named entities is a challenging research problem, but it has not received sufficient attention from the natural language processing community. In this short paper, we present our particip…
Data AugmentationEntity Linkingnamed-entity-recognitionNamed Entity Recognition+1CADEL: A Corpus of Administrative Web Documents for Japanese Entity Linking
Entity linking is the task of associating linguistic expressions with entries in a knowledge base that represent real-world entities and concepts. Language resources for this task have primarily been developed for Englis…
Entity DisambiguationEntity Linking