A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generation
Non-autoregressive (NAR) models simultaneously generate multiple outputs in a sequence, which significantly reduces the inference speed at the cost of accuracy drop compared to autoregressive baselines. Showing great potential for real-time applications, an increasing number of NAR models have been explored in different fields to mitigate the performance gap against AR models. In this work, we conduct a comparative study of various NAR modeling methods for end-to-end automatic speech recognition (ASR). Experiments are performed in the state-of-the-art setting using ESPnet. The results on various tasks provide interesting findings for developing an understanding of NAR ASR, such as the accuracy-speed trade-off and robustness against long-form utterances. We also show that the techniques can be combined for further improvement and applied to NAR end-to-end speech translation. All the implementations are publicly available to encourage further research in NAR speech processing.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionSpeech-to-TextText GenerationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding
Recently, there has been a growing interest in text-to-speech (TTS) methods that can be trained with minimal supervision by combining two types of discrete speech representations and using two sequence-to-sequence tasks …
Language ModelingLanguage ModellingmodelSpeech Synthesis+2Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study
In this work, we present an extensive study on the use of pre-trained language models for the task of automatic Counter Narrative (CN) generation to fight online hate speech in English. We first present a comparative stu…
Automatic Post-EditingLanguage ModelingLanguage ModellingMaximizing Mutual Information for Tacotron
End-to-end speech synthesis methods already achieve close-to-human quality performance. However compared to HMM-based and NN-based frame-to-frame regression methods, they are prone to some synthesis errors, such as missi…
AttributeSpeech SynthesisInvestigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translation
We study the possibilities of building a non-autoregressive speech-to-text translation model using connectionist temporal classification (CTC), and use CTC-based automatic speech recognition as an auxiliary task to impro…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition+3Continuous Speech Synthesis using per-token Latent Diffusion
The success of autoregressive transformer models with discrete tokens has inspired quantization-based approaches for continuous modalities, though these often limit reconstruction quality. We therefore introduce SALAD, a…
Image GenerationQuantizationSpeech Synthesistext-to-speech+1