paper-with-me

홈 › Papers

Highly Efficient 8-bit Low Precision Inference of Convolutional Neural Networks with IntelCaffe

2018-05-04 · Jiong Gong, Haihao Shen, Guoming Zhang, Xiaoli Liu, Shane Li, Ge Jin, Niharika Maheshwari, Evarist Fomenko, Eden Segal

High throughput and low latency inference of deep neural networks are critical for the deployment of deep learning applications. This paper presents the efficient inference techniques of IntelCaffe, the first Intel optimized deep learning framework that supports efficient 8-bit low precision inference and model optimization techniques of convolutional neural networks on Intel Xeon Scalable Processors. The 8-bit optimized model is automatically generated with a calibration process from FP32 model without the need of fine-tuning or retraining. We show that the inference throughput and latency with ResNet-50, Inception-v3 and SSD are improved by 1.38X-2.9X and 1.35X-3X respectively with neglectable accuracy loss from IntelCaffe FP32 baseline and by 56X-75X and 26X-37X from BVLC Caffe. All these techniques have been open-sourced on IntelCaffe GitHub1, and the artifact is provided to reproduce the result on Amazon AWS Cloud.

📄 PDF Abstract BibTeX arXiv:1805.08691

Code (1)

intel/caffe 공식 구현

Tasks

Deep LearningModel Optimization

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…
Non Maximum Suppression Non Maximum Suppression is a computer vision method that selects a single entity out of many overlapping entities (for example bounding boxes in object detection). The…
1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…
SSD SSD is a single-stage object detection method that discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios and scales per…

Similar Papers 제목 키워드 기반

HIGHLY EFFICIENT 8-BIT LOW PRECISION INFERENCE OF CONVOLUTIONAL NEURAL NETWORKS

2019-05-01 · ICLR 2019 5 · Haihao Shen, Jiong Gong, Xiaoli Liu, Guoming Zhang 외

High throughput and low latency inference of deep neural networks are critical for the deployment of deep learning applications. This paper presents a general technique toward 8-bit low precision inference of convolution…

image-classificationImage ClassificationImage Segmentationobject-detection+3

A Study of BFLOAT16 for Deep Learning Training

2019-05-29 · Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das 외

This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training across image classification, speech recogniti…

Deep Learningimage-classificationImage ClassificationLanguage Modeling+4

PositNN: Training Deep Neural Networks with Mixed Low-Precision Posit

2021-04-30 · Gonçalo Raposo, Pedro Tomás, Nuno Roma

Low-precision formats have proven to be an efficient way to reduce not only the memory footprint but also the hardware resources and power consumption of deep learning computations. Under this premise, the posit numerica…

HOBFLOPS CNNs: Hardware Optimized Bitslice-Parallel Floating-Point Operations for Convolutional Neural Networks

2020-07-11 · James Garland, David Gregg

Convolutional neural networks (CNNs) are typically trained using 16- or 32-bit floating-point (FP) and researchers show that low-precision floating-point (FP) can be highly effective for inference. Low-precision FP can b…

SeerNet: Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization

2019-06-01 · CVPR 2019 6 · Shijie Cao, Lingxiao Ma, Wencong Xiao, Chen Zhang 외

In this paper we present a novel and general method to accelerate convolutional neural network (CNN) inference by taking advantage of feature map sparsity. We experimentally demonstrate that a highly quantized version of…

Quantization