An Exploration of Length Generalization in Transformer-Based Speech Enhancement
The use of Transformer architectures has facilitated remarkable progress in speech enhancement. Training Transformers using substantially long speech utterances is often infeasible as self-attention suffers from quadratic complexity. It is a critical and unexplored challenge for a Transformer-based speech enhancement model to learn from short speech utterances and generalize to longer ones. In this paper, we conduct comprehensive experiments to explore the length generalization problem in speech enhancement with Transformer. Our findings first establish that position embedding provides an effective instrument to alleviate the impact of utterance length on Transformer-based speech enhancement. Specifically, we explore four different position embedding schemes to enable length generalization. The results confirm the superiority of relative position embeddings (RPEs) over absolute PE (APEs) in length generalization.
Code (0)
등록된 구현이 없습니다.
Tasks
PositionSpeech EnhancementMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploring Length Generalization For Transformer-based Speech Enhancement
Transformer network architecture has proven effective in speech enhancement. However, as its core module, self-attention suffers from quadratic complexity, making it infeasible for training on long speech utterances. In …
Speech EnhancementDiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors
Large-scale latent diffusion models (LDMs) excel in content generation across various modalities, but their reliance on phonemes and durations in text-to-speech (TTS) limits scalability and access from other fields. Whil…
text-to-speechText to SpeechRobust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speech
Autoregressive (AR) Transformer-based sequence models are known to have difficulty generalizing to sequences longer than those seen during training. When applied to text-to-speech (TTS), these models tend to drop or repe…
Decodertext-to-speechText to SpeechDPATD: Dual-Phase Audio Transformer for Denoising
Recent high-performance transformer-based speech enhancement models demonstrate that time domain methods could achieve similar performance as time-frequency domain methods. However, time-domain speech enhancement systems…
DenoisingSpeech EnhancementSelective State Space Model for Monaural Speech Enhancement
Voice user interfaces (VUIs) have facilitated the efficient interactions between humans and machines through spoken commands. Since real-word acoustic scenes are complex, speech enhancement plays a critical role for robu…
MambaSpeech Enhancement