Acoustic span embeddings for multilingual query-by-example search
Query-by-example (QbE) speech search is the task of matching spoken queries to utterances within a search collection. In low- or zero-resource settings, QbE search is often addressed with approaches based on dynamic time warping (DTW). Recent work has found that methods based on acoustic word embeddings (AWEs) can improve both performance and search speed. However, prior work on AWE-based QbE has primarily focused on English data and with single-word queries. In this work, we generalize AWE training to spans of words, producing acoustic span embeddings (ASE), and explore the application of ASE to QbE with arbitrary-length queries in multiple unseen languages. We consider the commonly used setting where we have access to labeled data in other languages (in our case, several low-resource languages) distinct from the unseen test languages. We evaluate our approach on the QUESST 2015 QbE tasks, finding that multilingual ASE-based search is much faster than DTW-based search and outperforms the best previously published results on this task.
Code (1)
Tasks
Dynamic Time WarpingWord EmbeddingsSimilar Papers 제목 키워드 기반
Leveraging multilingual transfer for unsupervised semantic acoustic word embeddings
Acoustic word embeddings (AWEs) are fixed-dimensional vector representations of speech segments that encode phonetic content so that different realisations of the same word have similar embeddings. In this paper we explo…
Word EmbeddingsWord SimilarityMultilingual acoustic word embeddings for zero-resource languages
This research addresses the challenge of developing speech applications for zero-resource languages that lack labelled data. It specifically uses acoustic word embedding (AWE) -- fixed-dimensional representations of vari…
Hate Speech DetectionKeyword SpottingWord EmbeddingsQuery-by-Example Search with Discriminative Neural Acoustic Word Embeddings
Query-by-example search often uses dynamic time warping (DTW) for comparing queries and proposed matching segments. Recent work has shown that comparing speech segments by representing them as fixed-dimensional vectors -…
Dynamic Time WarpingWord EmbeddingsNeural approaches to spoken content embedding
Comparing spoken segments is a central operation to speech processing. Traditional approaches in this area have favored frame-level dynamic programming algorithms, such as dynamic time warping, because they require no su…
Automatic Speech RecognitionDynamic Time Warpingspeech-recognitionSpeech Recognition+1Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Search
We propose to learn acoustic word embeddings with temporal context for query-by-example (QbE) speech search. The temporal context includes the leading and trailing word sequences of a word. We assume that there exist spo…
Dynamic Time WarpingTripletWord Embeddings