Variation Aware Training of Hybrid Precision Neural Networks with 28nm HKMG FeFET Based Synaptic Core

2022-02-21 · Sunanda Thunder, Po-Tsang Huang

This work proposes a hybrid-precision neural network training framework with an eNVM based computational memory unit executing the weighted sum operation and another SRAM unit, which stores the error in weight update during back propagation and the required number of pulses to update the weights in the hardware. The hybrid training algorithm for MLP based neural network with 28 nm ferroelectric FET (FeFET) as synaptic devices achieves inference accuracy up to 95% in presence of device and cycle variations. The architecture is primarily evaluated using behavioral or macro-model of FeFET devices with experimentally calibrated device variations and we have achieved accuracies compared to floating-point implementations.

📄 PDF Abstract BibTeX arXiv:2202.10912

Code (0)

등록된 구현이 없습니다.

Training Time Prediction for Mixed Precision-based Distributed Training

2026-04-17 · Minchul Kang, Changyong Shin, Jinwoo Jeong, Hyunho Lee 외 arxiv

Accurate prediction of training time in distributed deep learning is crucial for resource allocation, cost estimation, and job scheduling. We observe that the floating-point precision setting is a key determinant of trai…

Data Quality-aware Mixed-precision Quantization via Hybrid Reinforcement Learning

2023-02-09 · Yingchun Wang, Jingcai Guo, Song Guo, Weizhan Zhang

Mixed-precision quantization mostly predetermines the model bit-width settings before actual training due to the non-differential bit-width sampling process, obtaining sub-optimal performance. Worse still, the convention…

Quantization reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices

2024-07-02 · Juntao Zhao, Borui Wan, Yanghua Peng, Haibin Lin 외

A number of production deep learning clusters have attempted to explore inference hardware for DNN training, at the off-peak serving hours with many inference GPUs idling. Conducting DNN training with a combination of he…

GPU Quantization

HyQuant: Hybrid-Precision Quantization for LLM Attention

2026-08-28 · Jiatong Ding, Bingxin Xing, Yu Zhang, Dian Ding 외 arxiv

Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-bit quantization of the attention module often introduces large errors at very low bit-widths, causin…

HyBNN and FedHyBNN: (Federated) Hybrid Binary Neural Networks

2022-05-19 · Kinshuk Dua

Binary Neural Networks (BNNs), neural networks with weights and activations constrained to -1(0) and +1, are an alternative to deep neural networks which offer faster training, lower memory consumption and lightweight mo…

Binarization