Composite Binary Decomposition Networks
Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts. In this paper, we propose the composite binary decomposition networks (CBDNet), which first compose real-valued tensor of each layer with a limited number of binary tensors, and then decompose some conditioned binary tensors into two low-rank binary tensors, so that the number of parameters and operations are greatly reduced comparing to the original ones. Experiments demonstrate the effectiveness of the proposed method, as CBDNet can approximate image classification network ResNet-18 using 5.25 bits, VGG-16 using 5.47 bits, DenseNet-121 using 5.72 bits, object detection networks SSD300 using 4.38 bits, and semantic segmentation networks SegNet using 5.18 bits, all with minor accuracy drops.
Code (0)
등록된 구현이 없습니다.
Tasks
General Classificationimage-classificationImage Classificationobject-detectionObject DetectionSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Composite Multiclass Losses
We consider loss functions for multiclass prediction problems. We show when a multiclass loss can be expressed as a ``proper composite loss'', which is the composition of a proper loss and a link function. We exte…
General ClassificationSensitivityComposite Q-learning: Multi-scale Q-function Decomposition and Separable Optimization
In the past few years, off-policy reinforcement learning methods have shown promising results in their application for robot control. Deep Q-learning, however, still suffers from poor data-efficiency and is susceptible t…
Q-LearningReinforcement LearningProbabilistic modeling of discrete structural response with application to composite plate penetration models
Discrete response of structures is often a key probabilistic quantity of interest. For example, one may need to identify the probability of a binary event, such as, whether a structure has buckled or not. In this study, …
Storing cycles in Hopfield-type networks with pseudoinverse learning rule: admissibility and network topology
Cyclic patterns of neuronal activity are ubiquitous in animal nervous systems, and partially responsible for generating and controlling rhythmic movements such as locomotion, respiration, swallowing and so on. Clarifying…
Training Binary Weight Networks via Semi-Binary Decomposition
Recently binary weight networks have attracted lots of attentions due to their high computational efficiency and small parameter size. Yet they still suffer from large accuracy drops because of their limited representati…
Computational Efficiency