Probabilistic Blocking with An Application to the Syrian Conflict
Entity resolution seeks to merge databases as to remove duplicate entries where unique identifiers are typically unknown. We review modern blocking approaches for entity resolution, focusing on those based upon locality sensitive hashing (LSH). First, we introduce $k$-means locality sensitive hashing (KLSH), which is based upon the information retrieval literature and clusters similar records into blocks using a vector-space representation and projections. Second, we introduce a subquadratic variant of LSH to the literature, known as Densified One Permutation Hashing (DOPH). Third, we propose a weighted variant of DOPH. We illustrate each method on an application to a subset of the ongoing Syrian conflict, giving a discussion of each method.
Code (0)
등록된 구현이 없습니다.
Tasks
BlockingEntity ResolutionInformation RetrievalRetrievalSimilar Papers 제목 키워드 기반
Estimating the Effects of Syrian Civil War
We examine the effect of civil war in Syria on economic growth, human development and institutional quality. Building on the synthetic control method, we estimate the missing counterfactual scenario in the hypothetical a…
counterfactualHumanitarianStylistic Clusters and the Syrian/South Syrian Tradition of First-Millennium BCE Levantine Ivory Carving: A Machine Learning Approach
Thousands of first-millennium BCE ivory carvings have been excavated from Neo-Assyrian sites in Mesopotamia (primarily Nimrud, Khorsabad, and Arslan Tash) hundreds of miles from their Levantine production contexts. At pr…
BIG-bench Machine LearningDescriptiveDiagnosticGeneral ClassificationMonitoring War Destruction from Space: A Machine Learning Approach
Existing data on building destruction in conflict zones rely on eyewitness reports or manual detection, which makes it generally scarce, incomplete and potentially biased. This lack of reliable data imposes severe limita…
BIG-bench Machine LearningData AugmentationHumanitarianNabra: Syrian Arabic Dialects with Morphological Annotations
This paper presents Nabra, a corpora of Syrian Arabic dialects with morphological annotations. A team of Syrian natives collected more than 6K sentences containing about 60K words from several sources including social me…
SentenceNon-Blockingness Verification of Bounded Petri Nets Using Basis Reachability Graphs -- An Extended Version With Benchmarks
In this paper, we study the problem of non-blockingness verification by tapping into the basis reachability graph (BRG). Non-blockingness is a property that ensures that all pre-specified tasks can be completed, which is…