Structured Convolution Matrices for Energy-efficient Deep learning
We derive a relationship between network representation in energy-efficient neuromorphic architectures and block Toplitz convolutional matrices. Inspired by this connection, we develop deep convolutional networks using a family of structured convolutional matrices and achieve state-of-the-art trade-off between energy efficiency and classification accuracy for well-known image recognition tasks. We also put forward a novel method to train binary convolutional networks by utilising an existing connection between noisy-rectified linear units and binary activations.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningGeneral ClassificationSimilar Papers 제목 키워드 기반
Structured Weight Matrices-Based Hardware Accelerators in Deep Neural Networks: FPGAs and ASICs
Both industry and academia have extensively investigated hardware accelerations. In this work, to address the increasing demands in computational capability and memory requirement, we propose structured weight matrices (…
Learning Compressed Transforms with Low Displacement Rank
The low displacement rank (LDR) framework for structured matrices represents a matrix through two displacement operators and a low-rank residual. Existing use of LDR matrices in deep learning has applied fixed displaceme…
image-classificationImage ClassificationLanguage ModelingLanguage ModellingTraining Structured Efficient Convolutional Layers
Typical recent neural network designs are primarily convolutional layers, but the tricks enabling structured efficient linear layers (SELLs) have not yet been adapted to the convolutional setting. We present a method to …
Computational EfficiencyEfficient Recurrent Neural Networks using Structured Matrices in FPGAs
Recurrent Neural Networks (RNNs) are becoming increasingly important for time series-related applications which require efficient and real-time implementations. The recent pruning based work ESE suffers from degradation …
Model CompressionTime SeriesTime Series AnalysisSymmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks
There has been much recent interest in designing neural networks (NNs) with relaxed equivariance, which interpolate between exact equivariance and full flexibility for consistent performance gains. In a separate line of …