Binarized Weight Error Networks With a Transition Regularization Term
This paper proposes a novel binarized weight network (BT) for a resource-efficient neural structure. The proposed model estimates a binary representation of weights by taking into account the approximation error with an additional term. This model increases representation capacity and stability, particularly for shallow networks, while the computation load is theoretically reduced. In addition, a novel regularization term is introduced that is suitable for all threshold-based binary precision networks. This term penalizes the trainable parameters that are far from the thresholds at which binary transitions occur. This step promotes a swift modification for binary-precision responses at train time. The experimental results are carried out for two sets of tasks: visual classification and visual inverse problems. Benchmarks for Cifar10, SVHN, Fashion, ImageNet2012, Set5, Set14, Urban and BSD100 datasets show that our method outperforms all counterparts with binary precision.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
ProxyBNN: Learning Binarized Neural Networks via Proxy Matrices
Training Binarized Neural Networks (BNNs) is challenging due to the discreteness. In order to efficiently optimize BNNs through backward propagations, real-valued auxiliary variables are commonly used to accumulate gradi…
BinarizationQuantizationBDC-Occ: Binarized Deep Convolution Unit For Binarized Occupancy Network
Existing 3D occupancy networks demand significant hardware resources, hindering the deployment of edge devices. Binarized Neural Networks (BNN) offer substantially reduced computational and memory requirements. However, …
BinarizationTraining Experimentally Robust and Interpretable Binarized Regression Models Using Mixed-Integer Programming
In this paper, we explore model-based approach to training robust and interpretable binarized regression models for multiclass classification tasks using Mixed-Integer Programming (MIP). Our MIP model balances the optimi…
ClassificationregressionToward Extremely Low Bit and Lossless Accuracy in DNNs with Progressive ADMM
Weight quantization is one of the most important techniques of Deep Neural Networks (DNNs) model compression method. A recent work using systematic framework of DNN weight quantization with the advanced optimization algo…
Model CompressionQuantizationFast Binarized Neural Network Training with Partial Pre-training
Binarized neural networks, networks with weights and activations constrained to lie in a 2-element set, allow for more time- and resource-efficient inference than standard floating-point networks. However, binarized neur…