Expanding-and-Shrinking Binary Neural Networks
While binary neural networks (BNNs) offer significant benefits in terms of speed, memory and energy, they encounter substantial accuracy degradation in challenging tasks compared to their real-valued counterparts. Due to the binarization of weights and activations, the possible values of each entry in the feature maps generated by BNNs are strongly constrained. To tackle this limitation, we propose the expanding-and-shrinking operation, which enhances binary feature maps with negligible increase of computation complexity, thereby strengthening the representation capacity. Extensive experiments conducted on multiple benchmarks reveal that our approach generalizes well across diverse applications ranging from image classification, object detection to generative diffusion model, while also achieving remarkable improvement over various leading binarization algorithms based on different architectures including both CNNs and Transformers.
Code (1)
Tasks
Binarizationimage-classificationImage Classificationobject-detectionObject DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of Channels
One-Shot methods have evolved into one of the most popular methods in Neural Architecture Search (NAS) due to weight sharing and single training of a supernet. However, existing methods generally suffer from two issues: …
Neural Architecture SearchA new neural-network-based model for measuring the strength of a pseudorandom binary sequence
Maximum order complexity is an important tool for measuring the nonlinearity of a pseudorandom sequence. There is a lack of tools for predicting the strength of a pseudorandom binary sequence in an effective and efficien…
Incremental and Decremental Fuzzy Bounded Twin Support Vector Machine
In this paper we present an incremental variant of the Twin Support Vector Machine (TWSVM) called Fuzzy Bounded Twin Support Vector Machine (FBTWSVM) to deal with large datasets and learning from data streams. We combine…
General ClassificationRobust classificationMorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
We present MorphNet, an approach to automate the design of neural network structures. MorphNet iteratively shrinks and expands a network, shrinking via a resource-weighted sparsifying regularizer on activations and expan…
Neural Architecture SearchManifold Micro-Surgery with Linearly Nearly Euclidean Metrics
The Ricci flow is a method of manifold surgery, which can trim manifolds to more regular. However, in most cases, the Rich flow tends to develop singularities and lead to divergence of the solution. In this paper, we pro…