Lightweight Neural Networks
Most of the weights in a Lightweight Neural Network have a value of zero, while the remaining ones are either +1 or -1. These universal approximators require approximately 1.1 bits/weight of storage, posses a quick forward pass and achieve classification accuracies similar to conventional continuous-weight networks. Their training regimen focuses on error reduction initially, but later emphasizes discretization of weights. They ignore insignificant inputs, remove unnecessary weights, and drop unneeded hidden neurons. We have successfully tested them on the MNIST, credit card fraud, and credit card defaults data sets using networks having 2 to 16 hidden layers and up to 4.4 million weights.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationSimilar Papers 제목 키워드 기반
RepViT: Revisiting Mobile CNN From ViT Perspective
Recently, lightweight Vision Transformers (ViTs) demonstrate superior performance and lower latency, compared with lightweight Convolutional Neural Networks (CNNs), on resource-constrained mobile devices. Researchers hav…
Lightweight Image Super-Resolution with Multi-scale Feature Interaction Network
Recently, the single image super-resolution (SISR) approaches with deep and complex convolutional neural network structures have achieved promising performance. However, those methods improve the performance at the cost …
Image Super-ResolutionSuper-ResolutionPLDNet: PLD-Guided Lightweight Deep Network Boosted by Efficient Attention for Handheld Dual-Microphone Speech Enhancement
Low-complexity speech enhancement on mobile phones is crucial in the era of 5G. Thus, focusing on handheld mobile phone communication scenario, based on power level difference (PLD) algorithm and lightweight U-Net, we pr…
Speech EnhancementCSL-YOLO: A New Lightweight Object Detection System for Edge Computing
The development of lightweight object detectors is essential due to the limited computation resources. To reduce the computation cost, how to generate redundant features plays a significant role. This paper proposes a ne…
Edge-computingobject-detectionObject DetectionLightweight Backbone Networks Only Require Adaptive Lightweight Self-Attention Mechanisms
Currently, lightweight hybrid backbone networks have partially alleviated the issue of computational saturation, but the imbalance in computational efficiencys between convolutional neural networks (CNNs) and attention m…