Constructing Energy-efficient Mixed-precision Neural Networks through Principal Component Analysis for Edge Intelligence

2019-06-04 · Indranil Chakraborty, Deboleena Roy, Isha Garg, Aayush Ankit, Kaushik Roy

The `Internet of Things' has brought increased demand for AI-based edge computing in applications ranging from healthcare monitoring systems to autonomous vehicles. Quantization is a powerful tool to address the growing computational cost of such applications, and yields significant compression over full-precision networks. However, quantization can result in substantial loss of performance for complex image classification tasks. To address this, we propose a Principal Component Analysis (PCA) driven methodology to identify the important layers of a binary network, and design mixed-precision networks. The proposed Hybrid-Net achieves a more than 10% improvement in classification accuracy over binary networks such as XNOR-Net for ResNet and VGG architectures on CIFAR-100 and ImageNet datasets while still achieving up to 94% of the energy-efficiency of XNOR-Nets. This work furthers the feasibility of using highly compressed neural networks for energy-efficient neural computing in edge devices.

📄 PDF Abstract BibTeX arXiv:1906.01493

Code (1)

ichakra2/pca-hybrid 공식 구현 pytorch

Tasks

Autonomous Vehicles Dimensionality ReductionEdge-computingGeneral Classification image-classification Image Classification Quantization

Methods 이 논문이 사용한 방법론

Average Pooling 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Batch Normalization 설명 없음

Bottleneck Residual Block A Bottleneck Residual Block is a variant of the residual block that utilises 1x1 convolutions to create a bottleneck. The…

Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Residual Block Residual Blocks are skip-connection blocks that learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. They were introduced…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

BF-IMNA: A Bit Fluid In-Memory Neural Architecture for Neural Network Acceleration

2024-11-03 · Mariam Rakka, Rachid Karami, Ahmed M. Eltawil, Mohammed E. Fouda 외

Mixed-precision quantization works Neural Networks (NNs) are gaining traction for their efficient realization on the hardware leading to higher throughput and lower energy. In-Memory Computing (IMC) accelerator architect…

Quantization

Mixed-Precision Federated Learning via Multi-Precision Over-The-Air Aggregation

2024-06-04 · Jinsheng Yuan, Zhuangkun Wei, Weisi Guo

Over-the-Air Federated Learning (OTA-FL) is a privacy-preserving distributed learning mechanism, by aggregating updates in the electromagnetic channel rather than at the server. A critical research gap in existing OTA-FL…

Computational Efficiency Edge-computing Federated Learning Privacy Preserving+1

Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operations

2024-07-19 · Giorgos Armeniakos, Alexis Maras, Sotirios Xydis, Dimitrios Soudris

Recent advancements in quantization and mixed-precision approaches offers substantial opportunities to improve the speed and energy efficiency of Neural Networks (NN). Research has shown that individual parameters with v…

CPU Quantization

MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration

2025-09-18 · Giorgos Armeniakos, Alexis Maras, Sotirios Xydis, Dimitrios Soudris arxiv

The evolution of quantization and mixed-precision techniques has unlocked new possibilities for enhancing the speed and energy efficiency of NNs. Several recent studies indicate that adapting precision levels across diff…

Model Compression

Mixed-Precision Neural Networks: A Survey

2022-08-11 · Mariam Rakka, Mohammed E. Fouda, Pramod Khargonekar, Fadi Kurdahi

Mixed-precision Deep Neural Networks achieve the energy efficiency and throughput needed for hardware deployment, particularly when the resources are limited, without sacrificing accuracy. However, the optimal per-layer …

Quantization Reinforcement Learning (RL)Survey