Effect of shapes of activation functions on predictability in the echo state network
We investigate prediction accuracy for time series of Echo state networks with respect to several kinds of activation functions. As a result, we found that some kinds of activation functions with an appropriate nonlinearity show high performance compared to the conventional sigmoid function.
Code (0)
등록된 구현이 없습니다.
Tasks
Time SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
Fractal and Chaotic Activation Functions in Echo State Networks: Preprocessing Topology Governs the Echo State Property
Contemporary reservoir computing relies heavily on globally Lipschitz, well-behaved activation functions, limiting applications in defense, disaster response, and pharmaceutical modeling where robust operation under extr…
Universality and approximation bounds for echo state networks with random weights
We study the uniform approximation of echo state networks with randomly generated internal weights. These models, in which only the readout weights are optimized during training, have made empirical success in learning d…
Developing Training Procedures for Piecewise-linear Spline Activation Functions in Neural Networks
Activation functions in neural networks are typically selected from a set of empirically validated, commonly used static functions such as ReLU, tanh, or sigmoid. However, by optimizing the shapes of a network's activati…
Transformers with Learnable Activation Functions
Activation functions can have a significant impact on reducing the topological complexity of input data and therefore improve the performance of the model. Selecting a suitable activation function is an essential step in…
EchoMatch: Partial-to-Partial Shape Matching via Correspondence Reflection
Finding correspondences between 3D shapes is a crucial problem in computer vision and graphics. While most research has focused on finding correspondences in settings where at least one of the shapes is complete, the…