SpeechNAS: Towards Better Trade-off between Latency and Accuracy for Large-Scale Speaker Verification

2021-09-18 · Wentao Zhu, Tianlong Kong, Shun Lu, Jixiang Li, Dawei Zhang, Feng Deng, Xiaorui Wang, Sen yang, Ji Liu

Recently, x-vector has been a successful and popular approach for speaker verification, which employs a time delay neural network (TDNN) and statistics pooling to extract speaker characterizing embedding from variable-length utterances. Improvement upon the x-vector has been an active research area, and enormous neural networks have been elaborately designed based on the x-vector, eg, extended TDNN (E-TDNN), factorized TDNN (F-TDNN), and densely connected TDNN (D-TDNN). In this work, we try to identify the optimal architectures from a TDNN based search space employing neural architecture search (NAS), named SpeechNAS. Leveraging the recent advances in the speaker recognition, such as high-order statistics pooling, multi-branch mechanism, D-TDNN and angular additive margin softmax (AAM) loss with a minimum hyper-spherical energy (MHE), SpeechNAS automatically discovers five network architectures, from SpeechNAS-1 to SpeechNAS-5, of various numbers of parameters and GFLOPs on the large-scale text-independent speaker recognition dataset VoxCeleb1. Our derived best neural network achieves an equal error rate (EER) of 1.02% on the standard test set of VoxCeleb1, which surpasses previous TDNN based state-of-the-art approaches by a large margin. Code and trained weights are in https://github.com/wentaozhu/speechnas.git

📄 PDF Abstract BibTeX arXiv:2109.08839

Code (1)

wentaozhu/speechnas 공식 구현 pytorch

Tasks

Neural Architecture Search Speaker Recognition Speaker VerificationText-Independent Speaker Recognition

Methods 이 논문이 사용한 방법론

Test 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Joint Channel and Weight Pruning for Model Acceleration on Moblie Devices

2021-10-15 · Tianli Zhao, Xi Sheryl Zhang, Wentao Zhu, Jiaxing Wang 외

For practical deep neural network design on mobile devices, it is essential to consider the constraints incurred by the computational resources and the inference latency in various applications. Among deep network accele…

Model Compression

IPA: Inference Pipeline Adaptation to Achieve High Accuracy and Cost-Efficiency

2023-08-24 · Saeid Ghafouri, Kamran Razavi, Mehran Salmani, Alireza Sanaee 외

Efficiently optimizing multi-model inference pipelines for fast, accurate, and cost-effective inference is a crucial challenge in machine learning production systems, given their tight end-to-end latency requirements. To…

Deep Learning

MURMUR: An Efficient Inference System for Long-Form ASR

2026-05-31 · Wei-Tzu Lee, Keisuke Kamahori, Baris Kasikci arxiv

Long-form automatic speech recognition (ASR) requires both high accuracy and low latency, but existing systems force a trade-off between the two. Chunk-based pipelines process audio in parallel windows for low latency, b…

Speech Recognition

Learning Adaptive Segmentation Policy for Simultaneous Translation

2020-11-01 · EMNLP 2020 11 · Ruiqing Zhang, Chuanqiang Zhang, Zhongjun He, Hua Wu 외

Balancing accuracy and latency is a great challenge for simultaneous translation. To achieve high accuracy, the model usually needs to wait for more streaming text before translation, which results in increased latency. …

Segmentation Translation

NASH: Neural Architecture Search for Hardware-Optimized Machine Learning Models

2024-03-04 · Mengfei Ji, Yuchun Chang, Baolin Zhang, Zaid Al-Ars

As machine learning (ML) algorithms get deployed in an ever-increasing number of applications, these algorithms need to achieve better trade-offs between high accuracy, high throughput and low latency. This paper introdu…

Neural Architecture Search