Resource-Efficient Separation Transformer
Transformers have recently achieved state-of-the-art performance in speech separation. These models, however, are computationally demanding and require a lot of learnable parameters. This paper explores Transformer-based speech separation with a reduced computational cost. Our main contribution is the development of the Resource-Efficient Separation Transformer (RE-SepFormer), a self-attention-based architecture that reduces the computational burden in two ways. First, it uses non-overlapping blocks in the latent space. Second, it operates on compact latent summaries calculated from each chunk. The RE-SepFormer reaches a competitive performance on the popular WSJ0-2Mix and WHAM! datasets in both causal and non-causal settings. Remarkably, it scales significantly better than the previous Transformer-based architectures in terms of memory and inference time, making it more suitable for processing long mixtures.
Code (1)
Tasks
Speech SeparationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Papez: Resource-Efficient Speech Separation with Auditory Working Memory
Transformer-based models recently reached state-of-the-art single-channel speech separation accuracy; However, their extreme computational load makes it difficult to deploy them in resource-constrained mobile or IoT devi…
Speech SeparationSPMamba: State-space model is all you need in speech separation
Existing CNN-based speech separation models face local receptive field limitations and cannot effectively capture long time dependencies. Although LSTM and Transformer-based speech separation models can avoid this proble…
AllMambaSpeech SeparationUltra Fast Speech Separation Model with Teacher Student Learning
Transformer has been successfully applied to speech separation recently with its strong long-dependency modeling capacity using a self-attention mechanism. However, Transformer tends to have heavy run-time costs due to t…
Computational EfficiencySpeech SeparationPipeTransformer: Automated Elastic Pipelining for Distributed Training of Transformers
The size of Transformer models is growing at an unprecedented pace. It has only taken less than one year to reach trillion-level parameters after the release of GPT-3 (175B). Training such models requires both substantia…
Multi-Dimensional and Multi-Scale Modeling for Speech Separation Optimized by Discriminative Learning
Transformer has shown advanced performance in speech separation, benefiting from its ability to capture global features. However, capturing local features and channel information of audio sequences in speech separation i…
Speech Separation