Deep convolutional acoustic word embeddings using word-pair side information
Recent studies have been revisiting whole words as the basic modelling unit in speech recognition and query applications, instead of phonetic units. Such whole-word segmental systems rely on a function that maps a variable-length speech segment to a vector in a fixed-dimensional space; the resulting acoustic word embeddings need to allow for accurate discrimination between different word types, directly in the embedding space. We compare several old and new approaches in a word discrimination task. Our best approach uses side information in the form of known word pairs to train a Siamese convolutional neural network (CNN): a pair of tied networks that take two speech segments as input and produce their embeddings, trained with a hinge loss that separates same-word pairs and different-word pairs by some margin. A word classifier CNN performs similarly, but requires much stronger supervision. Both types of CNNs yield large improvements over the best previously published results on the word discrimination task.
Code (1)
Tasks
speech-recognitionSpeech RecognitionWord EmbeddingsSimilar Papers 제목 키워드 기반
Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Search
We propose to learn acoustic word embeddings with temporal context for query-by-example (QbE) speech search. The temporal context includes the leading and trailing word sequences of a word. We assume that there exist spo…
Dynamic Time WarpingTripletWord EmbeddingsNeural approaches to spoken content embedding
Comparing spoken segments is a central operation to speech processing. Traditional approaches in this area have favored frame-level dynamic programming algorithms, such as dynamic time warping, because they require no su…
Automatic Speech RecognitionDynamic Time Warpingspeech-recognitionSpeech Recognition+1Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approaches
Acoustic word embeddings --- fixed-dimensional vector representations of variable-length spoken word segments --- have begun to be considered for tasks such as speech recognition and query-by-example search. Such embeddi…
Dynamic Time WarpingGeneral Classificationspeech-recognitionSpeech Recognition+1Asymmetric Proxy Loss for Multi-View Acoustic Word Embeddings
Acoustic word embeddings (AWEs) are discriminative representations of speech segments, and learned embedding space reflects the phonetic similarity between words. With multi-view learning, where text labels are considere…
Metric LearningMULTI-VIEW LEARNINGTripletWord EmbeddingsMulti-view Recurrent Neural Acoustic Word Embeddings
Recent work has begun exploring neural acoustic word embeddings---fixed-dimensional vector representations of arbitrary-length speech segments corresponding to words. Such embeddings are applicable to speech retrieval an…
RetrievalWord EmbeddingsWord Similarity