Record Deduplication for Entity Distribution Modeling in ASR Transcripts
Voice digital assistants must keep up with trending search queries. We rely on a speech recognition model using contextual biasing with a rapidly updated set of entities, instead of frequent model retraining, to keep up with trends. There are several challenges with this approach: (1) the entity set must be frequently reconstructed, (2) the entity set is of limited size due to latency and accuracy trade-offs, and (3) finding the true entity distribution for biasing is complicated by ASR misrecognition. We address these challenges and define an entity set by modeling customers true requested entity distribution from ASR output in production using record deduplication, a technique from the field of entity resolution. Record deduplication resolves or deduplicates coreferences, including misrecognitions, of the same latent entity. Our method successfully retrieves 95% of misrecognized entities and when used for contextual biasing shows an estimated 5% relative word error rate reduction.
Code (0)
등록된 구현이 없습니다.
Tasks
Entity Resolutionspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
A Robust and Efficient Pipeline for Enterprise-Level Large-Scale Entity Resolution
Entity resolution (ER) remains a significant challenge in data management, especially when dealing with large datasets. This paper introduces MERAI (Massive Entity Resolution using AI), a robust and efficient pipeline de…
Entity ResolutionAn Ensemble Blocking Scheme for Entity Resolution of Large and Sparse Datasets
Entity Resolution, also called record linkage or deduplication, refers to the process of identifying and merging duplicate versions of the same entity into a unified representation. The standard practice is to use a Rule…
BlockingEntity ResolutionA Practioner's Guide to Evaluating Entity Resolution Results
Entity resolution (ER) is the task of identifying records belonging to the same entity (e.g. individual, group) across one or multiple databases. Ironically, it has multiple names: deduplication and record linkage, among…
ClusteringEntity ResolutionGeneral ClassificationMulti-class ClassificationSame Company, Same Signal: The Role of Identity in Earnings Call Transcripts
Post-earnings volatility prediction is critical for investors, with previous works often leveraging earnings call transcripts under the assumption that their rich semantics contribute significantly. To further investigat…
AttributeImproving Named Entity Recognition in Spoken Dialog Systems by Context and Speech Pattern Modeling
While named entity recognition (NER) from speech has been around as long as NER from written text has, the accuracy of NER from speech has generally been much lower than that of NER from text. The rise in popularity of s…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)named-entity-recognitionNamed Entity Recognition+4