홈 › Papers

RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition

2020-02-19 · Peiyan Dong, Siyue Wang, Wei Niu, Chengming Zhang, Sheng Lin, Zhengang Li, Yifan Gong, Bin Ren, Xue Lin, Yanzhi Wang, Dingwen Tao

Recurrent neural networks (RNNs) based automatic speech recognition has nowadays become prevalent on mobile devices such as smart phones. However, previous RNN compression techniques either suffer from hardware performance overhead due to irregularity or significant accuracy loss due to the preserved regularity for hardware friendliness. In this work, we propose RTMobile that leverages both a novel block-based pruning approach and compiler optimizations to accelerate RNN inference on mobile devices. Our proposed RTMobile is the first work that can achieve real-time RNN inference on mobile platforms. Experimental results demonstrate that RTMobile can significantly outperform existing RNN hardware acceleration methods in terms of inference accuracy and time. Compared with prior work on FPGA, RTMobile using Adreno 640 embedded GPU on GRU can improve the energy-efficiency by about 40$\times$ while maintaining the same inference time.

📄 PDF Abstract BibTeX arXiv:2002.11474

Code (0)

등록된 구현이 없습니다.

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)GPUspeech-recognition Speech Recognition

Methods 이 논문이 사용한 방법론

Pruning 설명 없음

GRU A Gated Recurrent Unit, or GRU, is a type of recurrent neural network. It is similar to an LSTM, but only has two gates - a reset…

NPAS: A Compiler-aware Framework of Unified Network Pruning and Architecture Search for Beyond Real-Time Mobile Acceleration

2020-12-01 · CVPR 2021 1 · Zhengang Li, Geng Yuan, Wei Niu, Pu Zhao 외

With the increasing demand to efficiently deploy DNNs on mobile edge devices, it becomes much more important to reduce unnecessary computation and increase the execution speed. Prior methods towards this goal, including …

Bayesian Optimization Code Generation Model Compression Network Pruning

Deep Learning Acceleration Techniques for Real Time Mobile Vision Applications

2019-05-09 · Gael Kamdem De Teyou

Deep Learning (DL) has become a crucial technology for Artificial Intelligence (AI). It is a powerful technique to automatically extract high-level features from complex data which can be exploited for applications such …

Deep Learning Distributed Computing object-detection Object Detection

GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity

2021-08-25 · Wei Niu, Zhengang Li, Xiaolong Ma, Peiyan Dong 외

It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices because even the powerful modern mobile devices are considered as ``resource-constrained'' when executing large-s…

Code Generation Compiler Optimization

RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices

2020-07-20 · Wei Niu, Mengshu Sun, Zhengang Li, Jou-An Chen 외

Mobile devices are becoming an important carrier for deep learning tasks, as they are being equipped with powerful, high-end mobile CPUs and GPUs. However, it is still a challenging task to execute 3D Convolutional Neura…

Code Generation Model Compression

Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration

2021-11-22 · Yifan Gong, Geng Yuan, Zheng Zhan, Wei Niu 외

Weight pruning is an effective model compression technique to tackle the challenges of achieving real-time deep neural network (DNN) inference on mobile devices. However, prior pruning schemes have limited application sc…

Model Compression