Integer Echo State Networks: Efficient Reservoir Computing for Digital Hardware
We propose an approximation of Echo State Networks (ESN) that can be efficiently implemented on digital hardware based on the mathematics of hyperdimensional computing. The reservoir of the proposed integer Echo State Network (intESN) is a vector containing only n-bits integers (where n<8 is normally sufficient for a satisfactory performance). The recurrent matrix multiplication is replaced with an efficient cyclic shift operation. The proposed intESN approach is verified with typical tasks in reservoir computing: memorizing of a sequence of inputs; classifying time-series; learning dynamic processes. Such architecture results in dramatic improvements in memory footprint and computational efficiency, with minimal performance loss. The experiments on a field-programmable gate array confirm that the proposed intESN approach is much more energy efficient than the conventional ESN.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational EfficiencyTime SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
Propagating Uncertainty through the tanh Function with Application to Reservoir Computing
Many neural networks use the tanh activation function, however when given a probability distribution as input, the problem of computing the output distribution in neural networks with tanh activation has not yet been add…
Scaling up Echo-State Networks with multiple light scattering
Echo-State Networks and Reservoir Computing have been studied for more than a decade. They provide a simpler yet powerful alternative to Recurrent Neural Networks, every internal weight is fixed and only the last linear …
Time SeriesTime Series AnalysisDeep-ESN: A Multiple Projection-encoding Hierarchical Reservoir Computing Framework
As an efficient recurrent neural network (RNN) model, reservoir computing (RC) models, such as Echo State Networks, have attracted widespread attention in the last decade. However, while they have had great success with …
Time SeriesTime Series AnalysisParalESN: Enabling parallel information processing in Reservoir Computing
Reservoir Computing (RC) has established itself as an efficient paradigm for temporal processing. However, its scalability remains severely constrained by the need to process temporal data sequentially and the prohibitiv…
Analysis of Wide and Deep Echo State Networks for Multiscale Spatiotemporal Time Series Forecasting
Echo state networks are computationally lightweight reservoir models inspired by the random projections observed in cortical circuitry. As interest in reservoir computing has grown, networks have become deeper and more i…
Time SeriesTime Series AnalysisTime Series Forecasting