Differentiable Weightless Neural Networks
We introduce the Differentiable Weightless Neural Network (DWN), a model based on interconnected lookup tables. Training of DWNs is enabled by a novel Extended Finite Difference technique for approximate differentiation of binary values. We propose Learnable Mapping, Learnable Reduction, and Spectral Regularization to further improve the accuracy and efficiency of these models. We evaluate DWNs in three edge computing contexts: (1) an FPGA-based hardware accelerator, where they demonstrate superior latency, throughput, energy efficiency, and model area compared to state-of-the-art solutions, (2) a low-power microcontroller, where they achieve preferable accuracy to XGBoost while subject to stringent memory constraints, and (3) ultra-low-cost chips, where they consistently outperform small models in both accuracy and projected hardware area. DWNs also compare favorably against leading approaches for tabular datasets, with higher average rank. Overall, our work positions DWNs as a pioneering solution for edge-compatible high-throughput neural networks.
Code (1)
Tasks
Edge-computingSimilar Papers 제목 키워드 기반
Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control
Controlling autonomous systems under real-world conditions often requires policies that can be evaluated with low latency and minimal energy consumption. Unfortunately, these conditions are at odds with the use of high-p…
Continuous ControlProcess mining classification with a weightless neural network
Using a weightless neural network architecture WiSARD we propose a straightforward graph to retina codification to represent business process graph flows avoiding kernels, and we present how WiSARD outperforms the classi…
ClassificationGeneral ClassificationWeightless Neural Networks for Continuously Trainable Personalized Recommendation Systems
Given that conventional recommenders, while deeply effective, rely on large distributed systems pre-trained on aggregate user data, incorporating new data necessitates large training cycles, making them slow to adapt to …
Collaborative FilteringRecommendation SystemsLearn Weightlessness: Imitate Non-Self-Stabilizing Motions on Humanoid Robot
The integration of imitation and reinforcement learning has enabled remarkable advances in humanoid whole-body control, facilitating diverse human-like behaviors. However, research on environment-dependent motions remain…
Reinforcement LearningMemory via Temporal Delays in weightless Spiking Neural Network
A common view in the neuroscience community is that memory is encoded in the connection strength between neurons. This perception led artificial neural network models to focus on connection weights as the key variables t…