Entity6K: A Large Open-Domain Evaluation Dataset for Real-World Entity Recognition
Open-domain real-world entity recognition is essential yet challenging, involving identifying various entities in diverse environments. The lack of a suitable evaluation dataset has been a major obstacle in this field due to the vast number of entities and the extensive human effort required for data curation. We introduce Entity6K, a comprehensive dataset for real-world entity recognition, featuring 5,700 entities across 26 categories, each supported by 5 human-verified images with annotations. Entity6K offers a diverse range of entity names and categorizations, addressing a gap in existing datasets. We conducted benchmarks with existing models on tasks like image captioning, object detection, zero-shot classification, and dense captioning to demonstrate Entity6K's effectiveness in evaluating models' entity recognition capabilities. We believe Entity6K will be a valuable resource for advancing accurate entity recognition in open-domain settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Dense CaptioningImage Captioningobject-detectionObject Detectionzero-shot-classificationZero-Shot LearningSimilar Papers 제목 키워드 기반
Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition
Open Named Entity Recognition (NER), which involves identifying arbitrary types of entities from arbitrary domains, remains challenging for Large Language Models (LLMs). Recent studies suggest that fine-tuning LLMs on ex…
DiversityDomain AdaptationDomain Generalizationnamed-entity-recognition+3OpenEL: An Annotated Corpus for Entity Linking and Discourse in Open Domain Dialogue
Entity linking in dialogue is the task of mapping entity mentions in utterances to a target knowledge base. Prior work on entity linking has mainly focused on well-written articles such as Wikipedia, annotated newswire, …
Articlescoreference-resolutionCoreference ResolutionEntity LinkingOpenLEAF: Open-Domain Interleaved Image-Text Generation and Evaluation
This work investigates a challenging task named open-domain interleaved image-text generation, which generates interleaved texts and images following an input query. We propose a new interleaved generation framework base…
Question AnsweringText GenerationEvaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP
Retrieval is a core component for open-domain NLP tasks. In open-domain tasks, multiple entities can share a name, making disambiguation an inherent yet under-explored problem. We propose an evaluation benchmark for asse…
Entity DisambiguationEntity RetrievalFact CheckingQuestion Answering+3Neural Correction Model for Open-Domain Named Entity Recognition
Named Entity Recognition (NER) plays an important role in a wide range of natural language processing tasks, such as relation extraction, question answering, etc. However, previous studies on NER are limited to particula…
Multi-Task Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+3