CircConv: A Structured Convolution with Low Complexity
Deep neural networks (DNNs), especially deep convolutional neural networks (CNNs), have emerged as the powerful technique in various machine learning applications. However, the large model sizes of DNNs yield high demands on computation resource and weight storage, thereby limiting the practical deployment of DNNs. To overcome these limitations, this paper proposes to impose the circulant structure to the construction of convolutional layers, and hence leads to circulant convolutional layers (CircConvs) and circulant CNNs. The circulant structure and models can be either trained from scratch or re-trained from a pre-trained non-circulant model, thereby making it very flexible for different training environments. Through extensive experiments, such strong structure-imposing approach is proved to be able to substantially reduce the number of parameters of convolutional layers and enable significant saving of computational cost by using fast multiplication of the circulant tensor.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Structured Convolutions for Efficient Neural Network Design
In this work, we tackle model efficiency by exploiting redundancy in the \textit{implicit structure} of the building blocks of convolutional neural networks. We start our analysis by introducing a general definition of C…
Efficient Neural NetworkImage ClassificationModel CompressionSemantic Segmentation+1Interleaved Structured Sparse Convolutional Neural Networks
In this paper, we study the problem of designing efficient convolutional neural network architectures with the interest in eliminating the redundancy in convolution kernels. In addition to structured sparse kernels, low-…
IGCV$2$: Interleaved Structured Sparse Convolutional Neural Networks
In this paper, we study the problem of designing efficient convolutional neural network architectures with the interest in eliminating the redundancy in convolution kernels. In addition to structured sparse kernels, low-…
Structured Sparsification with Joint Optimization of Group Convolution and Channel Shuffle
Recent advances in convolutional neural networks(CNNs) usually come with the expense of excessive computational overhead and memory footprint. Network compression aims to alleviate this issue by training compact models w…
Network PruningGraph Neural Networks on Quantum Computers
Graph Neural Networks (GNNs) are powerful machine learning models that excel at analyzing structured data represented as graphs, demonstrating remarkable performance in applications like social network analysis and recom…
Graph AttentionGraph Neural NetworkQuantum Machine LearningRecommendation Systems