Reducing hyperparameter dependence by external timescale tailoring
Task specific hyperparameter tuning in reservoir computing is an open issue, and is of particular relevance for hardware implemented reservoirs. We investigate the influence of directly including externally controllable task specific timescales on the performance and hyperparameter sensitivity of reservoir computing approaches. We show that the need for hyperparameter optimisation can be reduced if timescales of the reservoir are tailored to the specific task. Our results are mainly relevant for temporal tasks requiring memory of past inputs, for example chaotic timeseries prediciton. We consider various methods of including task specific timescales in the reservoir computing approach and demonstrate the universality of our message by looking at both time-multiplexed and spatially multiplexed reservoir computing.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
How to set AdamW's weight decay as you scale model and dataset size
The scaling of the optimal AdamW weight decay hyperparameter with model and dataset size is critical as we seek to build larger models, but is poorly understood. We show that weights learned by AdamW can be understood as…
Self-induced stochastic resonance: A physics-informed machine learning approach
Self-induced stochastic resonance (SISR) is the emergence of coherent oscillations in slow-fast excitable systems driven solely by noise, without external periodic forcing or proximity to a bifurcation. This work present…
Critical neuronal models with relaxed timescales separation
Power laws in nature are considered to be signatures of complexity. The theory of self-organized criticality (SOC) was proposed to explain their origins. A longstanding principle of SOC is the \emph{separation of timesca…
A two-step sequential approach for hyperparameter selection in finite context models
Finite-context models (FCMs) are widely used for compressing symbolic sequences such as DNA, where predictive performance depends critically on the context length k and smoothing parameter α. In practice, these hyperpara…
Mapping distinct timescales of functional interactions among brain networks
Brain processes occur at various timescales, ranging from milliseconds (neurons) to minutes and hours (behavior). Characterizing functional coupling among brain regions at these diverse timescales is key to understanding…
Functional ConnectivityRobust classification