Bridging the Gap between Reality and Ideality of Entity Matching: A Revisiting and Benchmark Re-Construction
Entity matching (EM) is the most critical step for entity resolution (ER). While current deep learningbased methods achieve very impressive performance on standard EM benchmarks, their realworld application performance is much frustrating. In this paper, we highlight that such the gap between reality and ideality stems from the unreasonable benchmark construction process, which is inconsistent with the nature of entity matching and therefore leads to biased evaluations of current EM approaches. To this end, we build a new EM corpus and re-construct EM benchmarks to challenge critical assumptions implicit in the previous benchmark construction process by step-wisely changing the restricted entities, balanced labels, and single-modal records in previous benchmarks into open entities, imbalanced labels, and multimodal records in an open environment. Experimental results demonstrate that the assumptions made in the previous benchmark construction process are not coincidental with the open environment, which conceal the main challenges of the task and therefore significantly overestimate the current progress of entity matching. The constructed benchmarks and code are publicly released
Code (0)
등록된 구현이 없습니다.
Tasks
Entity ResolutionSimilar Papers 제목 키워드 기반
Automated Reasoning in Normative Detachment Structures with Ideal Conditions
Systems of deontic logic suffer either from being too expressive and therefore hard to mechanize, or from being too simple to capture relevant aspects of normative reasoning. In this article we look for a suitable way in…
Learning Implicit Entity-object Relations by Bidirectional Generative Alignment for Multimodal NER
The challenge posed by multimodal named entity recognition (MNER) is mainly two-fold: (1) bridging the semantic gap between text and image and (2) matching the entity with its associated object in image. Existing methods…
named-entity-recognitionNamed Entity RecognitionNERObjectDomain-adaptive Person Re-identification without Cross-camera Paired Samples
Existing person re-identification (re-ID) research mainly focuses on pedestrian identity matching across cameras in adjacent areas. However, in reality, it is inevitable to face the problem of pedestrian identity matchin…
Domain Adaptive Person Re-IdentificationPerson Re-IdentificationNonideality-aware training makes memristive networks more robust to adversarial attacks
Neural networks are now deployed in a wide number of areas from object classification to natural language systems. Implementations using analog devices like memristors promise better power efficiency, potentially bringin…
Adversarial RobustnessBridging Imagination and Reality for Model-Based Deep Reinforcement Learning
Sample efficiency has been one of the major challenges for deep reinforcement learning. Recently, model-based reinforcement learning has been proposed to address this challenge by performing planning on imaginary traject…
Deep Reinforcement LearningModel-based Reinforcement Learningreinforcement-learningReinforcement Learning+1