ALF: Autoencoder-based Low-rank Filter-sharing for Efficient Convolutional Neural Networks
Closing the gap between the hardware requirements of state-of-the-art convolutional neural networks and the limited resources constraining embedded applications is the next big challenge in deep learning research. The computational complexity and memory footprint of such neural networks are typically daunting for deployment in resource constrained environments. Model compression techniques, such as pruning, are emphasized among other optimization methods for solving this problem. Most existing techniques require domain expertise or result in irregular sparse representations, which increase the burden of deploying deep learning applications on embedded hardware accelerators. In this paper, we propose the autoencoder-based low-rank filter-sharing technique technique (ALF). When applied to various networks, ALF is compared to state-of-the-art pruning methods, demonstrating its efficient compression capabilities on theoretical metrics as well as on an accurate, deterministic hardware-model. In our experiments, ALF showed a reduction of 70\% in network parameters, 61\% in operations and 41\% in execution time, with minimal loss in accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Model CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Joint Subspace View to Convolutional Neural Networks
Motivated by the intuition that important image regions remain important across different layers and scales in a CNN, we propose in this paper a joint subspace view to convolutional filters across network layers. When we…
Convexified Convolutional Neural Networks
We describe the class of convexified convolutional neural networks (CCNNs), which capture the parameter sharing of convolutional neural networks in a convex manner. By representing the nonlinear convolutional filters as …
DenoisingLearning Local Receptive Fields and their Weight Sharing Scheme on Graphs
We propose a simple and generic layer formulation that extends the properties of convolutional layers to any domain that can be described by a graph. Namely, we use the support of its adjacency matrix to design learnable…
Learning $3$D-FilterMap for Deep Convolutional Neural Networks
We present a novel and compact architecture for deep Convolutional Neural Networks (CNNs) in this paper, termed $3$D-FilterMap Convolutional Neural Networks ($3$D-FM-CNNs). The convolution layer of $3$D-FM-CNN learns a c…
Rank-1 Convolutional Neural Network
In this paper, we propose a convolutional neural network(CNN) with 3-D rank-1 filters which are composed by the outer product of 1-D filters. After being trained, the 3-D rank-1 filters can be decomposed into 1-D filters…