paper-with-me

Papers

A Fast Algorithm to Simulate Nonlinear Resistive Networks

2024-02-18 · Benjamin Scellier

Analog electrical networks have long been investigated as energy-efficient computing platforms for machine learning, leveraging analog physics during inference. More recently, resistor networks have sparked particular interest due to their ability to learn using local rules (such as equilibrium propagation), enabling potentially important energy efficiency gains for training as well. Despite their potential advantage, the simulations of these resistor networks has been a significant bottleneck to assess their scalability, with current methods either being limited to linear networks or relying on realistic, yet slow circuit simulators like SPICE. Assuming ideal circuit elements, we introduce a novel approach for the simulation of nonlinear resistive networks, which we frame as a quadratic programming problem with linear inequality constraints, and which we solve using a fast, exact coordinate descent algorithm. Our simulation methodology significantly outperforms existing SPICE-based simulations, enabling the training of networks up to 327 times larger at speeds 160 times faster, resulting in a 50,000-fold improvement in the ratio of network size to epoch duration. Our approach can foster more rapid progress in the simulations of nonlinear analog electrical networks.

📄 PDF Abstract BibTeX arXiv:2402.11674

Code (1)

rain-neuromorphics/energy-based-learning 공식 구현 pytorch

Similar Papers 제목 키워드 기반

An Operator-Theoretic Framework to Simulate Neuromorphic Circuits

2024-04-09 · Amir Shahhosseini, Thomas Chaffey, Rodolphe Sepulchre

Splitting algorithms are well-established in convex optimization and are designed to solve large-scale problems. Using such algorithms to simulate the behavior of nonlinear circuit networks provides scalable methods for …

Training End-to-End Analog Neural Networks with Equilibrium Propagation

2020-06-02 · Jack Kendall, Ross Pantone, Kalpana Manickavasagam, Yoshua Bengio 외

We introduce a principled method to train end-to-end analog neural networks by stochastic gradient descent. In these analog neural networks, the weights to be adjusted are implemented by the conductances of programmable …

Deep Neural Network Optimized to Resistive Memory with Nonlinear Current-Voltage Characteristics

2017-03-30 · Hyungjun Kim, Taesu Kim, Jinseok Kim, Jae-Joon Kim

Artificial Neural Network computation relies on intensive vector-matrix multiplications. Recently, the emerging nonvolatile memory (NVM) crossbar array showed a feasibility of implementing such operations with high energ…

Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network based on Analog Resistive Synapse

2017-12-16 · Chih-Cheng Chang, Pin-Chun Chen, Teyuh Chou, I-Ting Wang 외

Asymmetric nonlinear weight update is considered as one of the major obstacles for realizing hardware neural networks based on analog resistive synapses because it significantly compromises the online training capability…

General Classification

Training large-scale ANNs on simulated resistive crossbar arrays

2019-06-06 · Malte J. Rasch, Tayfun Gokmen, Wilfried Haensch

Accelerating training of artificial neural networks (ANN) with analog resistive crossbar arrays is a promising idea. While the concept has been verified on very small ANNs and toy data sets (such as MNIST), more realisti…