Evaluating Entity Linking: An Analysis of Current Benchmark Datasets and a Roadmap for Doing a Better Job
Entity linking has become a popular task in both natural language processing and semantic web communities. However, we find that the benchmark datasets for entity linking tasks do not accurately evaluate entity linking systems. In this paper, we aim to chart the strengths and weaknesses of current benchmark datasets and sketch a roadmap for the community to devise better benchmark datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Entity LinkingSimilar Papers 제목 키워드 기반
CADEL: A Corpus of Administrative Web Documents for Japanese Entity Linking
Entity linking is the task of associating linguistic expressions with entries in a knowledge base that represent real-world entities and concepts. Language resources for this task have primarily been developed for Englis…
Entity DisambiguationEntity LinkingMELO: An Evaluation Benchmark for Multilingual Entity Linking of Occupations
We present the Multilingual Entity Linking of Occupations (MELO) Benchmark, a new collection of 48 datasets for evaluating the linking of entity mentions in 21 languages to the ESCO Occupations multilingual taxonomy. MEL…
Entity LinkingSentenceDynamicER: Resolving Emerging Mentions to Dynamic Entities for RAG
In the rapidly evolving landscape of language, resolving new linguistic expressions in continuously updating knowledge bases remains a formidable challenge. This challenge becomes critical in retrieval-augmented generati…
Entity LinkingRAGRetrievalRetrieval-augmented GenerationInvestigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
A typical architecture for end-to-end entity linking systems consists of three steps: mention detection, candidate generation and entity disambiguation. In this study we investigate the following questions: (a) Can all t…
Entity DisambiguationEntity LinkingMachine TranslationQuestion Answering+3AMELI: Enhancing Multimodal Entity Linking with Fine-Grained Attributes
We propose attribute-aware multimodal entity linking, where the input is a mention described with a text and image, and the goal is to predict the corresponding target entity from a multimodal knowledge base (KB) where e…
AttributeEntity Linking