Machine Learning Techniques to Construct Patched Analog Ensembles for Data Assimilation
Using generative models from the machine learning literature to create artificial ensemble members for use within data assimilation schemes has been introduced in [Grooms QJRMS, 2020] as constructed analog ensemble optimal interpolation (cAnEnOI). Specifically, we study general and variational autoencoders for the machine learning component of this method, and combine the ideas of constructed analogs and ensemble optimal interpolation in the data assimilation piece. To extend the scalability of cAnEnOI for use in data assimilation on complex dynamical models, we propose using patching schemes to divide the global spatial domain into digestible chunks. Using patches makes training the generative models possible and has the added benefit of being able to exploit parallelism during the generative step. Testing this new algorithm on a 1D toy model, we find that larger patch sizes make it harder to train an accurate generative model (i.e. a model whose reconstruction error is small), while conversely the data assimilation performance improves at larger patch sizes. There is thus a sweet spot where the patch size is large enough to enable good data assimilation performance, but not so large that it becomes difficult to train an accurate generative model. In our tests the new patched cAnEnOI method outperforms the original (unpatched) cAnEnOI, as well as the ensemble square root filter results from [Grooms QJRMS, 2020].
Code (1)
Tasks
BIG-bench Machine LearningSimilar Papers 제목 키워드 기반
Clustering of Indonesian and Western Gamelan Orchestras through Machine Learning of Performance Parameters
Indonesian and Western gamelan ensembles are investigated with respect to performance differences. Thereby, the often exotistic history of this music in the West might be reflected in contemporary tonal system, articulat…
FormUncovering the Limits of Machine Learning for Automatic Vulnerability Detection
Recent results of machine learning for automatic vulnerability detection (ML4VD) have been very promising. Given only the source code of a function $f$, ML4VD techniques can decide if $f$ contains a security flaw with up…
BenchmarkingData AugmentationOut-of-Distribution GeneralizationVulnerability DetectionTreant: Training Evasion-Aware Decision Trees
Despite its success and popularity, machine learning is now recognized as vulnerable to evasion attacks, i.e., carefully crafted perturbations of test inputs designed to force prediction errors. In this paper we focus on…
BIG-bench Machine LearningEmergence of Structure in Ensembles of Random Neural Networks
Randomness is ubiquitous in many applications across data science and machine learning. Remarkably, systems composed of random components often display emergent global behaviors that appear deterministic, manifesting a t…
An Adaptive Synaptic Array using Fowler-Nordheim Dynamic Analog Memory
In this paper we present a synaptic array that uses dynamical states to implement an analog memory for energy-efficient training of machine learning (ML) systems. Each of the analog memory elements is a micro-dynamical s…