paper-with-me

Papers

Approximating Continuous Convolutions for Deep Network Compression

2022-10-17 · Theo W. Costain, Victor Adrian Prisacariu

We present ApproxConv, a novel method for compressing the layers of a convolutional neural network. Reframing conventional discrete convolution as continuous convolution of parametrised functions over space, we use functional approximations to capture the essential structures of CNN filters with fewer parameters than conventional operations. Our method is able to reduce the size of trained CNN layers requiring only a small amount of fine-tuning. We show that our method is able to compress existing deep network models by half whilst losing only 1.86% accuracy. Further, we demonstrate that our method is compatible with other compression methods like quantisation allowing for further reductions in model size.

📄 PDF Abstract BibTeX arXiv:2210.08951

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Similar Papers 제목 키워드 기반

Convolutional Filtering on Sampled Manifolds

2022-11-20 · Zhiyang Wang, Luana Ruiz, Alejandro Ribeiro

The increasing availability of geometric data has motivated the need for information processing over non-Euclidean domains modeled as manifolds. The building block for information processing architectures with desirable …

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

2022-02-04 · Mingguo He, Zhewei Wei, Ji-Rong Wen

Designing spectral convolutional networks is a challenging problem in graph learning. ChebNet, one of the early attempts, approximates the spectral graph convolutions using Chebyshev polynomials. GCN simplifies ChebNet b…

GPRGraph LearningNode Classification

QuadConv: Quadrature-Based Convolutions with Applications to Non-Uniform PDE Data Compression

2022-11-09 · Kevin Doherty, Cooper Simpson, Stephen Becker, Alireza Doostan

We present a new convolution layer for deep learning architectures which we call QuadConv -- an approximation to continuous convolution via quadrature. Our operator is developed explicitly for use on non-uniform, mesh-ba…

Data Compression

Low-Rank Compression of Neural Nets: Learning the Rank of Each Layer

2020-06-01 · CVPR 2020 6 · Yerlan Idelbayev, Miguel A. Carreira-Perpinan

Neural net compression can be achieved by approximating each layer's weight matrix by a low-rank matrix. The real difficulty in doing this is not in training the resulting neural net (made up of one low-rank matrix per l…

Low-rank compression

Deep $k$-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions

2018-06-24 · Junru Wu, Yue Wang, Zhen-Yu Wu, Zhangyang Wang 외

The current trend of pushing CNNs deeper with convolutions has created a pressing demand to achieve higher compression gains on CNNs where convolutions dominate the computation and parameter amount (e.g., GoogLeNet, ResN…

Clustering