Alternating Weak Triphone/BPE Alignment Supervision from Hybrid Model Improves End-to-End ASR

2024-02-23 · Jintao Jiang, Yingbo Gao, Mohammad Zeineldeen, Zoltan Tuske

In this paper, alternating weak triphone/BPE alignment supervision is proposed to improve end-to-end model training. Towards this end, triphone and BPE alignments are extracted using a pre-existing hybrid ASR system. Then, regularization effect is obtained by cross-entropy based intermediate auxiliary losses computed on such alignments at a mid-layer representation of the encoder for triphone alignments and at the encoder for BPE alignments. Weak supervision is achieved through strong label smoothing with parameter of 0.5. Experimental results on TED-LIUM 2 indicate that either triphone or BPE alignment based weak supervision improves ASR performance over standard CTC auxiliary loss. Moreover, their combination lowers the word error rate further. We also investigate the alternation of the two auxiliary tasks during model training, and additional performance gain is observed. Overall, the proposed techniques result in over 10% relative error rate reduction over a CTC-regularized baseline system.

📄 PDF Abstract BibTeX arXiv:2402.15594

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Weak Alignment Supervision from Hybrid Model Improves End-to-end ASR

2023-11-24 · Jintao Jiang, Yingbo Gao, Zoltan Tuske

In this paper, we aim to create weak alignment supervision from an existing hybrid system to aid the end-to-end modeling of automatic speech recognition. Towards this end, we use the existing hybrid ASR system to produce…

Automatic Speech Recognition speech-recognition Speech Recognition

Deep Triphone Embedding Improves Phoneme Recognition

2017-10-22 · Mohit Yadav, Vivek Tyagi

In this paper, we present a novel Deep Triphone Embedding (DTE) representation derived from Deep Neural Network (DNN) to encapsulate the discriminative information present in the adjoining speech frames. DTEs are generat…

Dimensionality Reduction General Classification Phoneme Recognition

H2FA R-CNN: Holistic and Hierarchical Feature Alignment for Cross-Domain Weakly Supervised Object Detection

2022-01-01 · CVPR 2022 1 · Yunqiu Xu, Yifan Sun, Zongxin Yang, Jiaxu Miao 외

Cross-domain weakly supervised object detection (CDWSOD) aims to adapt the detection model to a novel target domain with easily acquired image-level annotations. How to align the source and target domains is critical…

Domain Adaptation object-detection Object Detection Weakly Supervised Object Detection

Improving Weak-to-Strong Generalization with Reliability-Aware Alignment

2024-06-27 · Yue Guo, Yi Yang

Large language models (LLMs) are now rapidly advancing and surpassing human abilities on many natural language tasks. However, aligning these super-human LLMs with human knowledge remains challenging because the supervis…

4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling

2023-11-29 · CVPR 2024 1 · Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein 외

Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes. However, current text-to-4D methods face a three-way tradeoff between the quality of…