Raw Waveform Encoder with Multi-Scale Globally Attentive Locally Recurrent Networks for End-to-End Speech Recognition

2021-06-08 · Max W. Y. Lam, Jun Wang, Chao Weng, Dan Su, Dong Yu

End-to-end speech recognition generally uses hand-engineered acoustic features as input and excludes the feature extraction module from its joint optimization. To extract learnable and adaptive features and mitigate information loss, we propose a new encoder that adopts globally attentive locally recurrent (GALR) networks and directly takes raw waveform as input. We observe improved ASR performance and robustness by applying GALR on different window lengths to aggregate fine-grain temporal information into multi-scale acoustic features. Experiments are conducted on a benchmark dataset AISHELL-2 and two large-scale Mandarin speech corpus of 5,000 hours and 21,000 hours. With faster speed and comparable model size, our proposed multi-scale GALR waveform encoder achieved consistent character error rate reductions (CERRs) from 7.9% to 28.1% relative over strong baselines, including Conformer and TDNN-Conformer. In particular, our approach demonstrated notable robustness than the traditional handcrafted features and outperformed the baseline MFCC-based TDNN-Conformer model by a 15.2% CERR on a music-mixed real-world speech test set.

📄 PDF Abstract BibTeX arXiv:2106.04275

Code (0)

등록된 구현이 없습니다.

Tasks

speech-recognition Speech Recognition

Y-Vector: Multiscale Waveform Encoder for Speaker Embedding

2020-10-24 · Ge Zhu, Fei Jiang, Zhiyao Duan

State-of-the-art text-independent speaker verification systems typically use cepstral features or filter bank energies as speech features. Recent studies attempted to extract speaker embeddings directly from raw waveform…

Speaker Verification Text-Independent Speaker Verification

Motion-Attentive Transition for Zero-Shot Video Object Segmentation

2020-03-09 · Tianfei Zhou, Shunzhou Wang, Yi Zhou, Yazhou Yao 외

In this paper, we present a novel Motion-Attentive Transition Network (MATNet) for zero-shot video object segmentation, which provides a new way of leveraging motion information to reinforce spatio-temporal object repres…

Decoder Object Segmentation Semantic Segmentation+4

Modelling Sentence Pairs with Tree-structured Attentive Encoder

2016-10-10 · COLING 2016 12 · Yao Zhou, Cong Liu, Yan Pan

We describe an attentive encoder that combines tree-structured recursive neural networks and sequential recurrent neural networks for modelling sentence pairs. Since existing attentive models exert attention on the seque…

Paraphrase Identification Question Selection Semantic Similarity Semantic Textual Similarity+1

SL-S4Wave: Self-Supervised Learning of Physiological Waveforms with Structured State Space Models

2026-06-18 · Feng Wu, Harsh Deep, Eric Lehman, Sanyam Kapoor 외 arxiv

Modeling long-sequence medical time series data, such as electrocardiograms (ECG), poses significant challenges due to high sampling rates, multichannel signal complexity, inherent noise, and limited labeled data. While …

Self-Supervised Learning Domain Generalization Arrhythmia Detection Contrastive Learning

SA-EMO: Structure-Aligned Encoder Mixture of Operators for Generalizable Full-waveform Inversion

2025-11-07 · Wang Zhenyu, Li Peiyuan, Shi Yongxiang, Wu Ruoyu 외 arxiv

Full-waveform inversion (FWI) can produce high-resolution subsurface models, yet it remains inherently ill-posed, highly nonlinear, and computationally intensive. Although recent deep learning and numerical acceleration …