Binary Input Layer: Training of CNN models with binary input data
For the efficient execution of deep convolutional neural networks (CNN) on edge devices, various approaches have been presented which reduce the bit width of the network parameters down to 1 bit. Binarization of the first layer was always excluded, as it leads to a significant error increase. Here, we present the novel concept of binary input layer (BIL), which allows the usage of binary input data by learning bit specific binary weights. The concept is evaluated on three datasets (PAMAP2, SVHN, CIFAR-10). Our results show that this approach is in particular beneficial for multimodal datasets (PAMAP2) where it outperforms networks using full precision weights in the first layer by 1:92 percentage points (pp) while consuming only 2 % of the chip area.
Code (0)
등록된 구현이 없습니다.
Tasks
BinarizationSimilar Papers 제목 키워드 기반
Training Multi-Layer Binary Neural Networks With Local Binary Error Signals
Binary Neural Networks (BNNs) significantly reduce computational complexity and memory usage in machine and deep learning by representing weights and activations with just one bit. However, most existing training algorit…
Multi-class ClassificationBinary autoencoder with random binary weights
Here is presented an analysis of an autoencoder with binary activations $\{0, 1\}$ and binary $\{0, 1\}$ random weights. Such set up puts this model at the intersection of different fields: neuroscience, information theo…
ReStoCNet: Residual Stochastic Binary Convolutional Spiking Neural Network for Memory-Efficient Neuromorphic Computing
In this work, we propose ReStoCNet, a residual stochastic multilayer convolutional Spiking Neural Network (SNN) composed of binary kernels, to reduce the synaptic memory footprint and enhance the computational efficiency…
Computational EfficiencyDimensionality ReductionA Learning-Inspired Strategy to Design Binary Sequences with Good Correlation Properties: SISO and MIMO Radar Systems
In this paper, the design of binary sequences exhibiting low values of aperiodic/periodic correlation functions, in terms of Integrated Sidelobe Level (ISL), is pursued via a learning-inspired method. Specifcally, the sy…
Iterative Training: Finding Binary Weight Deep Neural Networks with Layer Binarization
In low-latency or mobile applications, lower computation complexity, lower memory footprint and better energy efficiency are desired. Many prior works address this need by removing redundant parameters. Parameter quantiz…
BinarizationQuantization