Automated Metadata Harmonization Using Entity Resolution & Contextual Embedding
ML Data Curation process typically consist of heterogeneous & federated source systems with varied schema structures; requiring curation process to standardize metadata from different schemas to an inter-operable schema. This manual process of Metadata Harmonization & cataloging slows efficiency of ML-Ops lifecycle. We demonstrate automation of this step with the help of entity resolution methods & also by using Cogntive Database's Db2Vec embedding approach to capture hidden inter-column & intra-column relationships which detect similarity of metadata and then predict metadata columns from source schemas to any standardized schemas. Apart from matching schemas, we demonstrate that it can also infer the correct ontological structure of the target data model.
Code (0)
등록된 구현이 없습니다.
Tasks
Entity ResolutionSimilar Papers 제목 키워드 기반
Identity resolution of software metadata using Large Language Models
Software is an essential component of research. However, little attention has been paid to it compared with that paid to research data. Recently, there has been an increase in efforts to acknowledge and highlight the imp…
FairnessMetadata-Aligned 3D MRI Representations for Contrast Understanding and Quality Control
Magnetic Resonance Imaging suffers from substantial data heterogeneity and the absence of standardized contrast labels across scanners, protocols, and institutions, which severely limits large-scale automated analysis. A…
Harmonizing Metadata of Language Resources for Enhanced Querying and Accessibility
This paper addresses the harmonization of metadata from diverse repositories of language resources (LRs). Leveraging linked data and RDF techniques, we integrate data from multiple sources into a unified model based on D…
High-Resolution Image Harmonization via Collaborative Dual Transformations
Given a composite image, image harmonization aims to adjust the foreground to make it compatible with the background. High-resolution image harmonization is in high demand, but still remains unexplored. Conventional imag…
Image HarmonizationVocal Bursts Intensity PredictionBeyond MR Image Harmonization: Resolution Matters Too
Magnetic resonance (MR) imaging is commonly used in the clinical setting to non-invasively monitor the body. There exists a large variability in MR imaging due to differences in scanner hardware, software, and protocol d…
Image HarmonizationSuper-Resolution