Training Progressively Binarizing Deep Networks Using FPGAs
While hardware implementations of inference routines for Binarized Neural Networks (BNNs) are plentiful, current realizations of efficient BNN hardware training accelerators, suitable for Internet of Things (IoT) edge devices, leave much to be desired. Conventional BNN hardware training accelerators perform forward and backward propagations with parameters adopting binary representations, and optimization using parameters adopting floating or fixed-point real-valued representations--requiring two distinct sets of network parameters. In this paper, we propose a hardware-friendly training method that, contrary to conventional methods, progressively binarizes a singular set of fixed-point network parameters, yielding notable reductions in power and resource utilizations. We use the Intel FPGA SDK for OpenCL development environment to train our progressively binarizing DNNs on an OpenVINO FPGA. We benchmark our training approach on both GPUs and FPGAs using CIFAR-10 and compare it to conventional BNNs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs
Mapping neural networks to FPGAs enables low-latency, energy-efficient inference, particularly for lookup table (LUT)-based models that eliminate multipliers and map directly to reconfigurable fabric. While prior work ac…
The Feasibility of Implementing Large-Scale Transformers on Multi-FPGA Platforms
FPGAs are rarely mentioned when discussing the implementation of large machine learning applications, such as Large Language Models (LLMs), in the data center. There has been much evidence showing that single FPGAs can b…
Hardware/Software Codesign for Training/Testing Multiple Neural Networks on Multiple FPGAs
Most neural network designs for FPGAs are inflexible. In this paper, we propose a flexible VHDL structure that would allow any neural network to be implemented on multiple FPGAs. Moreover, the VHDL structure allows for t…
On the Resilience of Deep Learning for Reduced-voltage FPGAs
Deep Neural Networks (DNNs) are inherently computation-intensive and also power-hungry. Hardware accelerators such as Field Programmable Gate Arrays (FPGAs) are a promising solution that can satisfy these requirements fo…
Deep LearningSelf-Binarizing Networks
We present a method to train self-binarizing neural networks, that is, networks that evolve their weights and activations during training to become binary. To obtain similar binary networks, existing methods rely on the …
Binarization