홈 › Papers

A Compression-Compilation Framework for On-mobile Real-time BERT Applications

2021-05-30 · Wei Niu, Zhenglun Kong, Geng Yuan, Weiwen Jiang, Jiexiong Guan, Caiwen Ding, Pu Zhao, Sijia Liu, Bin Ren, Yanzhi Wang

Transformer-based deep learning models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks. In this paper, we propose a compression-compilation co-design framework that can guarantee the identified model to meet both resource and real-time specifications of mobile devices. Our framework applies a compiler-aware neural architecture optimization method (CANAO), which can generate the optimal compressed model that balances both accuracy and latency. We are able to achieve up to 7.8x speedup compared with TensorFlow-Lite with only minor accuracy loss. We present two types of BERT applications on mobile devices: Question Answering (QA) and Text Generation. Both can be executed in real-time with latency as low as 45ms. Videos for demonstrating the framework can be found on https://www.youtube.com/watch?v=_WIRvK_2PZI

📄 PDF Abstract BibTeX arXiv:2106.00526

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering Text Generation

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Residual Connection 설명 없음

WordPiece 설명 없음

YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design

2020-09-12 · Yuxuan Cai, Hongjia Li, Geng Yuan, Wei Niu 외

The rapid development and wide utilization of object detection techniques have aroused attention on both accuracy and speed of object detectors. However, the current state-of-the-art object detection works are either acc…

Computational Efficiency CPU GPU Object+3

CoCoPIE: Making Mobile AI Sweet As PIE --Compression-Compilation Co-Design Goes a Long Way

2020-03-14 · Shaoshan Liu, Bin Ren, Xipeng Shen, Yanzhi Wang

Assuming hardware is the major constraint for enabling real-time mobile intelligence, the industry has mainly dedicated their efforts to developing specialized hardware accelerators for machine learning and inference. Th…

Latent Context Compilation: Distilling Long Context into Compact Portable Memory

2026-01-31 · Zeju Li, Yizhou Zhou, Qiang Xu arxiv

Efficient long-context LLM deployment is stalled by a dichotomy between amortized compression, which struggles with out-of-distribution generalization, and Test-Time Training, which incurs prohibitive synthetic data cost…

AdaSpring: Context-adaptive and Runtime-evolutionary Deep Model Compression for Mobile Applications

2021-01-28 · Sicong Liu, Bin Guo, Ke Ma, Zhiwen Yu 외

There are many deep learning (e.g., DNN) powered mobile and wearable applications today continuously and unobtrusively sensing the ambient surroundings to enhance all aspects of human lives. To enable robust and private …

Model Compression

MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices?

2026-03-12 · Xingze Zou, Jing Wang, Yuhua Zheng, Xueyi Chen 외 arxiv

Large language models (LLMs) have demonstrated remarkable capabilities in code generation, yet their potential for generating kernels specifically for mobile devices remains largely unexplored. In this work, we extend th…

Code Generation