Acoustically Grounded Word Embeddings for Improved Acoustics-to-Word Speech Recognition
Direct acoustics-to-word (A2W) systems for end-to-end automatic speech recognition are simpler to train, and more efficient to decode with, than sub-word systems. However, A2W systems can have difficulties at training time when data is limited, and at decoding time when recognizing words outside the training vocabulary. To address these shortcomings, we investigate the use of recently proposed acoustic and acoustically grounded word embedding techniques in A2W systems. The idea is based on treating the final pre-softmax weight matrix of an AWE recognizer as a matrix of word embedding vectors, and using an externally trained set of word embeddings to improve the quality of this matrix. In particular we introduce two ideas: (1) Enforcing similarity at training time between the external embeddings and the recognizer weights, and (2) using the word embeddings at test time for predicting out-of-vocabulary words. Our word embedding model is acoustically grounded, that is it is learned jointly with acoustic embeddings so as to encode the words' acoustic-phonetic content; and it is parametric, so that it can embed any arbitrary (potentially out-of-vocabulary) sequence of characters. We find that both techniques improve the performance of an A2W recognizer on conversational telephone speech.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionWord EmbeddingsSimilar Papers 제목 키워드 기반
Neural approaches to spoken content embedding
Comparing spoken segments is a central operation to speech processing. Traditional approaches in this area have favored frame-level dynamic programming algorithms, such as dynamic time warping, because they require no su…
Automatic Speech RecognitionDynamic Time Warpingspeech-recognitionSpeech Recognition+1Whole-Word Segmental Speech Recognition with Acoustic Word Embeddings
Segmental models are sequence prediction models in which scores of hypotheses are based on entire variable-length segments of frames. We consider segmental models for whole-word ("acoustic-to-word") speech recognition, w…
GPUspeech-recognitionSpeech RecognitionWord EmbeddingsMultilingual Jointly Trained Acoustic and Written Word Embeddings
Acoustic word embeddings (AWEs) are vector representations of spoken word segments. AWEs can be learned jointly with embeddings of character sequences, to generate phonetically meaningful embeddings of written words, or …
Dynamic Time WarpingRetrievalWord EmbeddingsAsymmetric Proxy Loss for Multi-View Acoustic Word Embeddings
Acoustic word embeddings (AWEs) are discriminative representations of speech segments, and learned embedding space reflects the phonetic similarity between words. With multi-view learning, where text labels are considere…
Metric LearningMULTI-VIEW LEARNINGTripletWord EmbeddingsEmoInstruct-TTS: Dual-Path Instruction-Guided Emotional Speech Synthesis
Instruction-based controllable speech synthesis enables users to specify emotions through natural language. However, existing approaches often rely on coarse emotion labels and lack explicit modeling of fine-grained inte…
Speech Synthesis