Hardware-efficient Residual Networks for FPGAs
Residual networks (ResNets) employ skip connections in their networks -- reusing activations from previous layers -- to improve training convergence, but these skip connections create challenges for hardware implementations of ResNets. The hardware must either wait for skip connections to be processed before processing more incoming data or buffer them elsewhere. Without skip connections, ResNets would be more hardware-efficient. Thus, we present the teacher-student learning method to gradually prune away all of a ResNet's skip connections, constructing a network we call NonResNet. We show that when implemented for FPGAs, NonResNet decreases ResNet's BRAM utilization by 9% and LUT utilization by 3% and increases throughput by 5%.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Semi-Streaming Architecture: A New Design Paradigm for CNN Implementation on FPGAs
The recent research advances in deep learning have led to the development of small and powerful Convolutional Neural Network (CNN) architectures. Meanwhile Field Programmable Gate Arrays (FPGAs) has become a popular hard…
A General Neural Network Hardware Architecture on FPGA
Field Programmable Gate Arrays (FPGAs) plays an increasingly important role in data sampling and processing industries due to its highly parallel architecture, low power consumption, and flexibility in custom algorithms.…
Autonomous DrivingBIG-bench Machine LearningCaffe Barista: Brewing Caffe with FPGAs in the Training Loop
As the complexity of deep learning (DL) models increases, their compute requirements increase accordingly. Deploying a Convolutional Neural Network (CNN) involves two phases: training and inference. With the inference ta…
Accelerating CNN inference on FPGAs: A Survey
Convolutional Neural Networks (CNNs) are currently adopted to solve an ever greater number of problems, ranging from speech recognition to image classification and segmentation. The large amount of processing required by…
Diversityimage-classificationImage Classificationspeech-recognition+2Unsupervised Recycled FPGA Detection Using Symmetry Analysis
Recently, recycled field-programmable gate arrays (FPGAs) pose a significant hardware security problem due to the proliferation of the semiconductor supply chain. Ring oscillator (RO) based frequency analyzing technique …
Anomaly DetectionDensity Ratio EstimationUnsupervised Anomaly Detection