How to Evaluate Entity Resolution Systems: An Entity-Centric Framework with Application to Inventor Name Disambiguation
Entity resolution (record linkage, microclustering) systems are notoriously difficult to evaluate. Looking for a needle in a haystack, traditional evaluation methods use sophisticated, application-specific sampling schemes to find matching pairs of records among an immense number of non-matches. We propose an alternative that facilitates the creation of representative, reusable benchmark data sets without necessitating complex sampling schemes. These benchmark data sets can then be used for model training and a variety of evaluation tasks. Specifically, we propose an entity-centric data labeling methodology that integrates with a unified framework for monitoring summary statistics, estimating key performance metrics such as cluster and pairwise precision and recall, and analyzing root causes for errors. We validate the framework in an application to inventor name disambiguation and through simulation studies. Software: https://github.com/OlivierBinette/er-evaluation/
Code (1)
Tasks
Entity ResolutionSimilar Papers 제목 키워드 기반
Adapting an Entity Centric Model for Portuguese Coreference Resolution
This paper presents the adaptation of an Entity Centric Model for Portuguese coreference resolution, considering 10 named entity categories. The model was evaluated on named e using the HAREM Portuguese corpus and the re…
coreference-resolutionCoreference ResolutionEntity-Centric Joint Modeling of Japanese Coreference Resolution and Predicate Argument Structure Analysis
Predicate argument structure analysis is a task of identifying structured events. To improve this field, we need to identify a salient entity, which cannot be identified without performing coreference resolution and pred…
coreference-resolutionCoreference ResolutionReading ComprehensionImproving Coreference Resolution by Leveraging Entity-Centric Features with Graph Neural Networks and Second-order Inference
One of the major challenges in coreference resolution is how to make use of entity-level features defined over clusters of mentions rather than mention pairs. However, coreferent mentions usually spread far apart in an e…
coreference-resolutionCoreference ResolutionGraph Neural Network