Self-Supervised Evolution Operator Learning for High-Dimensional Dynamical Systems
We introduce an encoder-only approach to learn the evolution operators of large-scale non-linear dynamical systems, such as those describing complex natural phenomena. Evolution operators are particularly well-suited for analyzing systems that exhibit complex spatio-temporal patterns and have become a key analytical tool across various scientific communities. As terabyte-scale weather datasets and simulation tools capable of running millions of molecular dynamics steps per day are becoming commodities, our approach provides an effective tool to make sense of them from a data-driven perspective. The core of it lies in a remarkable connection between self-supervised representation learning methods and the recently established learning theory of evolution operators. To show the usefulness of the proposed method, we test it across multiple scientific domains: explaining the folding dynamics of small proteins, the binding process of drug-like molecules in host sites, and autonomously finding patterns in climate data. Code and data to reproduce the experiments are made available open source.
Code (1)
Tasks
Learning TheoryOperator learningRepresentation LearningSimilar Papers 제목 키워드 기반
Evolutionary Augmentation Policy Optimization for Self-supervised Learning
Self-supervised Learning (SSL) is a machine learning algorithm for pretraining Deep Neural Networks (DNNs) without requiring manually labeled data. The central idea of this learning technique is based on an auxiliary sta…
Data AugmentationSelf-Supervised LearningRedefining Neural Operators in $d+1$ Dimensions
Neural Operators have emerged as powerful tools for learning mappings between function spaces. Among them, the kernel integral operator has been widely validated on universally approximating various operators. Although r…
Super-ResolutionLearn to Evolve: Self-supervised Neural JKO Operator for Wasserstein Gradient Flow
The Jordan-Kinderlehrer-Otto (JKO) scheme provides a stable variational framework for computing Wasserstein gradient flows, but its practical use is often limited by the high computational cost of repeatedly solving the …
Data AugmentationThe evolution of cooperation: an evolutionary advantage of individuals impedes the evolution of the population
Range expansion is a universal process in biological systems, and therefore plays a part in biological evolution. Using a quantitative individual-based method based on the stochastic process, we identify that enhancing t…
Evolution: A Unified Formula for Feature Operators from a High-level Perspective
Traditionally, different types of feature operators (e.g., convolution, self-attention and involution) utilize different approaches to extract and aggregate the features. Resemblance can be hardly discovered from their m…