Relational Proxy Loss for Audio-Text based Keyword Spotting
In recent years, there has been an increasing focus on user convenience, leading to increased interest in text-based keyword enrollment systems for keyword spotting (KWS). Since the system utilizes text input during the enrollment phase and audio input during actual usage, we call this task audio-text based KWS. To enable this task, both acoustic and text encoders are typically trained using deep metric learning loss functions, such as triplet- and proxy-based losses. This study aims to improve existing methods by leveraging the structural relations within acoustic embeddings and within text embeddings. Unlike previous studies that only compare acoustic and text embeddings on a point-to-point basis, our approach focuses on the relational structures within the embedding space by introducing the concept of Relational Proxy Loss (RPL). By incorporating RPL, we demonstrated improved performance on the Wall Street Journal (WSJ) corpus.
Code (0)
등록된 구현이 없습니다.
Tasks
Keyword SpottingMetric LearningTripletMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting
In this paper, we propose a novel end-to-end user-defined keyword spotting method that utilizes linguistically corresponding patterns between speech and text sequences. Unlike previous approaches requiring speech keyword…
Keyword SpottingContrastive Learning With Audio Discrimination For Customizable Keyword Spotting In Continuous Speech
Customizable keyword spotting (KWS) in continuous speech has attracted increasing attention due to its real-world application potential. While contrastive learning (CL) has been widely used to extract keyword representat…
Contrastive LearningKeyword SpottingText MatchingBridging the Gap between Audio and Text using Parallel-attention for User-defined Keyword Spotting
This paper proposes a novel user-defined keyword spotting framework that accurately detects audio keywords based on text enrollment. Since audio data possesses additional acoustic information compared to text, there are …
Keyword SpottingJoint Multimodal Contrastive Learning for Robust Spoken Term Detection and Keyword Spotting
Acoustic Word Embeddings (AWEs) improve the efficiency of speech retrieval tasks such as Spoken Term Detection (STD) and Keyword Spotting (KWS). However, existing approaches suffer from limitations, including unimodal su…
Contrastive LearningKeyword SpottingMATE: Matryoshka Audio-Text Embeddings for Open-Vocabulary Keyword Spotting
Open-vocabulary keyword spotting (KWS) with text-based enrollment has emerged as a flexible alternative to fixed-phrase triggers. Prior utterance-level matching methods, from an embedding-learning standpoint, learn embed…
Keyword SpottingMetric Learning