MASS-UMAP: Fast and accurate analog ensemble search in weather radar archive
The use of analogs - similar weather patterns - for weather forecasting and analysis is an established method in meteorology. The most challenging aspect of using this approach in the context of operational radar applications is to be able to perform a fast and accurate search for similar spatiotemporal precipitation patterns in a large archive of historical records. In this context, sequential pairwise search is too slow and computationally expensive. Here we propose an architecture to significantly speed-up spatiotemporal analog retrieval by combining nonlinear geometric dimensionality reduction (UMAP) with the fastest known Euclidean search algorithm for time series (MASS) to find radar analogs in constant time, independently of the desired temporal length to match and the number of extracted analogs. We compare UMAP with Principal component analysis (PCA) and show that UMAP outperforms PCA for spatial MSE analog search with proper settings. Moreover, we show that MASS is 20 times faster than brute force search on the UMAP embeddings space. We test the architecture on a real dataset and show that it enables precise and fast operational analog ensemble search through more than 2 years of radar archive in less than 5 seconds on a single workstation.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionRetrievalTime SeriesTime Series AnalysisWeather ForecastingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bringing UMAP Closer to the Speed of Light with GPU Acceleration
The Uniform Manifold Approximation and Projection (UMAP) algorithm has become widely popular for its ease of use, quality of results, and support for exploratory, unsupervised, supervised, and semi-supervised learning. W…
GPUFastUMAP: Scalable Dimensionality Reduction via Bipartite Landmark Sampling
Exploratory analysis of high-dimensional data rarely stops at a single embedding. In practice, analysts rerun dimensionality reduction after changing preprocessing, subsets, or hyperparameters, and standard nonlinear met…
Dimensionality ReductionUnveiling 3D Ocean Biogeochemical Provinces: A Machine Learning Approach for Systematic Clustering and Validation
Defining ocean regions and water masses helps to understand marine processes and can serve downstream-tasks such as defining marine protected areas. However, such definitions are often a result of subjective decisions po…
ClusteringGiDR-DUN; Gradient Dimensionality Reduction -- Differences and Unification
TSNE and UMAP are two of the most popular dimensionality reduction algorithms due to their speed and interpretable low-dimensional embeddings. However, while attempts have been made to improve on TSNE's computational com…
Dimensionality ReductionExtremely-Fast, Energy-Efficient Massive MIMO Precoding with Analog RRAM Matrix Computing
Signal processing in wireless communications, such as precoding, detection, and channel estimation, are basically about solving inverse matrix problems, which, however, are slow and inefficient in conventional digital co…