홈 › Papers

Adaptive Feature Selection for End-to-End Speech Translation

2020-10-16 · Findings of the Association for Computational Linguistics 2020 · Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich

Information in speech signals is not evenly distributed, making it an additional challenge for end-to-end (E2E) speech translation (ST) to learn to focus on informative features. In this paper, we propose adaptive feature selection (AFS) for encoder-decoder based E2E ST. We first pre-train an ASR encoder and apply AFS to dynamically estimate the importance of each encoded speech feature to SR. A ST encoder, stacked on top of the ASR encoder, then receives the filtered features from the (frozen) ASR encoder. We take L0DROP (Zhang et al., 2020) as the backbone for AFS, and adapt it to sparsify speech features with respect to both temporal and feature dimensions. Results on LibriSpeech En-Fr and MuST-C benchmarks show that AFS facilitates learning of ST by pruning out ~84% temporal features, yielding an average translation gain of ~1.3-1.6 BLEU and a decoding speedup of ~1.4x. In particular, AFS reduces the performance gap compared to the cascade baseline, and outperforms it on LibriSpeech En-Fr with a BLEU score of 18.56 (without data augmentation)

📄 PDF Abstract BibTeX arXiv:2010.08518

Code (1)

bzhangGo/zero 공식 구현 tf

Tasks

Data Augmentation Decoderfeature selectionTranslation

Methods 이 논문이 사용한 방법론

Pruning 설명 없음

Feature Selection Feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features (variables,…

Beyond Sentence-Level End-to-End Speech Translation: Context Helps

2021-08-01 · ACL 2021 5 · Biao Zhang, Ivan Titov, Barry Haddow, Rico Sennrich

Document-level contextual information has shown benefits to text-based machine translation, but whether and how context helps end-to-end (E2E) speech translation (ST) is still under-studied. We fill this gap through exte…

Computational Efficiency feature selection Machine Translation Sentence+1

Analyzing Speech Unit Selection for Textless Speech-to-Speech Translation

2024-07-08 · Jarod Duret, Yannick Estève, Titouan Parcollet

Recent advancements in textless speech-to-speech translation systems have been driven by the adoption of self-supervised learning techniques. Although most state-of-the-art systems adopt a similar architecture to transfo…

Automatic Speech Recognition Emotion Recognition feature selection Resynthesis+7

Edinburgh’s End-to-End Multilingual Speech Translation System for IWSLT 2021

2021-08-01 · ACL (IWSLT) 2021 8 · Biao Zhang, Rico Sennrich

This paper describes Edinburgh’s submissions to the IWSLT2021 multilingual speech translation (ST) task. We aim at improving multilingual translation and zero-shot performance in the constrained setting (without using an…

Data Augmentation feature selection Machine Translation Multi-Task Learning+2

Learning Adaptive Segmentation Policy for End-to-End Simultaneous Translation

2022-05-01 · ACL 2022 5 · Ruiqing Zhang, Zhongjun He, Hua Wu, Haifeng Wang

End-to-end simultaneous speech-to-text translation aims to directly perform translation from streaming source speech to target text with high translation quality and low latency. A typical simultaneous translation (ST) s…

Segmentation Simultaneous Speech-to-Text Translation Speech-to-Text Speech-to-Text Translation+1

Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training

2020-10-20 · Findings of the Association for Computational Linguistics 2020 · Renjie Zheng, Mingbo Ma, Baigong Zheng, Kaibo Liu 외

Simultaneous speech-to-speech translation is widely useful but extremely challenging, since it needs to generate target-language speech concurrently with the source-language speech, with only a few seconds delay. In addi…

Sentence Simultaneous Speech-to-Speech Translation Speech-to-Speech Translation Translation