A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems
We introduce a score-filter-enhanced data assimilation framework designed to reduce predictive uncertainty in machine learning (ML) models for data-driven dynamical system forecasting. Machine learning serves as an efficient numerical model for predicting dynamical systems. However, even with sufficient data, model uncertainty remains and accumulates over time, causing the long-term performance of ML models to deteriorate. To overcome this difficulty, we integrate data assimilation techniques into the training process to iteratively refine the model predictions by incorporating observational information. Specifically, we apply the Ensemble Score Filter (EnSF), a generative AI-based training-free diffusion model approach, for solving the data assimilation problem in high-dimensional nonlinear complex systems. This leads to a hybrid data assimilation-training framework that combines ML with EnSF to improve long-term predictive performance. We shall demonstrate that EnSF-enhanced ML can effectively reduce predictive uncertainty in ML-based Lorenz-96 system prediction and the Korteweg-De Vries (KdV) equation prediction.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A competitive baseline for deep learning enhanced data assimilation using conditional Gaussian ensemble Kalman filtering
Ensemble Kalman Filtering (EnKF) is a popular technique for data assimilation, with far ranging applications. However, the vanilla EnKF framework is not well-defined when perturbations are nonlinear. We study two non-lin…
Flow Matching for Efficient and Scalable Data Assimilation
Data assimilation (DA) estimates a dynamical system's state from noisy observations. Recent generative models like the ensemble score filter (EnSF) improve DA in high-dimensional nonlinear settings but are computationall…
Fire-EnSF: Wildfire Spread Data Assimilation using Ensemble Score Filter
As wildfires become increasingly destructive and expensive to control, effective management of active wildfires requires accurate, real-time fire spread predictions. To enhance the forecasting accuracy of active fires, d…
Computational EfficiencyAuto-differentiable data assimilation: Co-learning of states, dynamics, and filtering algorithms
Data assimilation algorithms estimate the state of a dynamical system from partial observations, where the successful performance of these algorithms hinges on costly parameter tuning and on employing an accurate model f…
Integrating Score-Based Diffusion Models with Machine Learning-Enhanced Localization for Advanced Data Assimilation in Geological Carbon Storage
Accurate characterization of subsurface heterogeneity is important for the safe and effective implementation of geological carbon storage (GCS) projects. This paper explores how machine learning methods can enhance data …