Evaluation of LLMs on Long-tail Entity Linking in Historical Documents
Entity Linking (EL) plays a crucial role in Natural Language Processing (NLP) applications, enabling the disambiguation of entity mentions by linking them to their corresponding entries in a reference knowledge base (KB). Thanks to their deep contextual understanding capabilities, LLMs offer a new perspective to tackle EL, promising better results than traditional methods. Despite the impressive generalization capabilities of LLMs, linking less popular, long-tail entities remains challenging as these entities are often underrepresented in training data and knowledge bases. Furthermore, the long-tail EL task is an understudied problem, and limited studies address it with LLMs. In the present work, we assess the performance of two popular LLMs, GPT and LLama3, in a long-tail entity linking scenario. Using MHERCL v0.1, a manually annotated benchmark of sentences from domain-specific historical texts, we quantitatively compare the performance of LLMs in identifying and linking entities to their corresponding Wikidata entries against that of ReLiK, a state-of-the-art Entity Linking and Relation Extraction framework. Our preliminary experiments reveal that LLMs perform encouragingly well in long-tail EL, indicating that this technology can be a valuable adjunct in filling the gap between head and long-tail EL.
Code (0)
등록된 구현이 없습니다.
Tasks
Entity LinkingRelation ExtractionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Systematic Study of Long Tail Phenomena in Entity Linking
State-of-the-art entity linkers achieve high accuracy scores with probabilistic methods. However, these scores should be considered in relation to the properties of the datasets they are evaluated on. Until now, there ha…
Entity LinkingLLMAEL: Large Language Models are Good Context Augmenters for Entity Linking
Entity Linking (EL) models are well-trained at mapping mentions to their corresponding entities according to a given context. However, EL models struggle to disambiguate long-tail entities due to their limited training d…
Data AugmentationEntity LinkingEmpirical Evaluation of Pretraining Strategies for Supervised Entity Linking
In this work, we present an entity linking model which combines a Transformer architecture with large scale pretraining from Wikipedia links. Our model achieves the state-of-the-art on two commonly used entity linking da…
Entity LinkingOneNet: A Fine-Tuning Free Framework for Few-Shot Entity Linking via Large Language Model Prompting
Entity Linking (EL) is the process of associating ambiguous textual mentions to specific entities in a knowledge base. Traditional EL methods heavily rely on large datasets to enhance their performance, a dependency that…
Entity LinkingFew-Shot LearningHallucinationLanguage Modeling+2Zero-shot Cross-Language Transfer of Monolingual Entity Linking Models
Most entity linking systems, whether mono or multilingual, link mentions to a single English knowledge base. Few have considered linking non-English text to a non-English KB, and therefore, transferring an English entity…
Cross-Lingual TransferEntity LinkingZero-Shot Cross-Lingual Transfer