Scalable Mechanistic Neural Networks for Differential Equations and Machine Learning
We propose Scalable Mechanistic Neural Network (S-MNN), an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences. By reformulating the original Mechanistic Neural Network (MNN) (Pervez et al., 2024), we reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear. This significant improvement enables efficient modeling of long-term dynamics without sacrificing accuracy or interpretability. Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources. Consequently, S-MNN can drop-in replace the original MNN in applications, providing a practical and efficient tool for integrating mechanistic bottlenecks into neural network models of complex dynamical systems. Source code is available at https://github.com/IST-DASLab/ScalableMNN.
Code (1)
Tasks
Temporal SequencesSimilar Papers 제목 키워드 기반
Mechanistic Neural Networks for Scientific Machine Learning
This paper presents Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing …
Equation DiscoveryGPUMechanistic PDE Networks for Discovery of Governing Equations
We present Mechanistic PDE Networks -- a model for discovery of governing partial differential equations from data. Mechanistic PDE Networks represent spatiotemporal data as space-time dependent linear partial differenti…
GPUAssessment of Uncertainty Quantification in Universal Differential Equations
Scientific Machine Learning is a new class of approaches that integrate physical knowledge and mechanistic models with data-driven techniques for uncovering governing equations of complex processes. Among the available a…
Uncertainty QuantificationVariational InferenceEARS-UDE: Evaluating Auditory Response in Sensory Overload with Universal Differential Equations
Auditory sensory overload affects 50-70% of individuals with Autism Spectrum Disorder (ASD), yet existing approaches, such as mechanistic models (Hodgkin Huxley type, Wilson Cowan, excitation inhibition balance), clinica…
Graph Neural Ordinary Differential Equations for Coarse-Grained Socioeconomic Dynamics
We present a data-driven machine-learning approach for modeling space-time socioeconomic dynamics. Through coarse-graining fine-scale observations, our modeling framework simplifies these complex systems to a set of trac…
Sensitivity