A Dual-Decoder Conformer for Multilingual Speech Recognition
Transformer-based models have recently become very popular for sequence-to-sequence applications such as machine translation and speech recognition. This work proposes a dual-decoder transformer model for low-resource multilingual speech recognition for Indian languages. Our proposed model consists of a Conformer [1] encoder, two parallel transformer decoders, and a language classifier. We use a phoneme decoder (PHN-DEC) for the phoneme recognition task and a grapheme decoder (GRP-DEC) to predict grapheme sequence along with language information. We consider phoneme recognition and language identification as auxiliary tasks in the multi-task learning framework. We jointly optimize the network for phoneme recognition, grapheme recognition, and language identification tasks with Joint CTC-Attention [2] training. Our experiments show that we can obtain a significant reduction in WER over the baseline approaches. We also show that our dual-decoder approach obtains significant improvement over the single decoder approach.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderLanguage IdentificationMachine TranslationMulti-Task LearningPhoneme Recognitionspeech-recognitionSpeech RecognitionTranslationSimilar Papers 제목 키워드 기반
Multilingual Speech Recognition for Low-Resource Indian Languages using Multi-Task conformer
Transformers have recently become very popular for sequence-to-sequence applications such as machine translation and speech recognition. In this work, we propose a multi-task learning-based transformer model for low-reso…
DecoderMachine TranslationMulti-Task LearningPhoneme Recognition+3Decoder-only Conformer with Modality-aware Sparse Mixtures of Experts for ASR
We present a decoder-only Conformer for automatic speech recognition (ASR) that processes speech and text in a single stack without external speech encoders or pretrained large language models (LLM). The model uses a mod…
Speech RecognitionText GenerationCanary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and AST
This report introduces Canary-1B-v2, a fast, robust multilingual model for Automatic Speech Recognition (ASR) and Speech-to-Text Translation (AST). Built with a FastConformer encoder and Transformer decoder, it supports …
Speech-to-Text TranslationSpeech RecognitionFast Conformer with Linearly Scalable Attention for Efficient Speech Recognition
Conformer-based models have become the dominant end-to-end architecture for speech processing tasks. With the objective of enhancing the conformer architecture for efficient training and inference, we carefully redesigne…
Automatic Speech RecognitionDecoderspeech-recognitionSpeech Recognition+2Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer
Humans are adept at leveraging visual cues from lip movements for recognizing speech in adverse listening conditions. Audio-Visual Speech Recognition (AVSR) models follow similar approach to achieve robust speech recogni…
Audio-Visual Speech RecognitionRobust Speech Recognitionspeech-recognitionSpeech Recognition+1