WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models
In recent years, AI-based weather forecasting models have matched or even outperformed numerical weather prediction systems. However, most of these models have been trained and evaluated on reanalysis datasets like ERA5. These datasets, being products of numerical models, often diverge substantially from actual observations in some crucial variables like near-surface temperature, wind, precipitation and clouds - parameters that hold significant public interest. To address this divergence, we introduce WeatherReal, a novel benchmark dataset for weather forecasting, derived from global near-surface in-situ observations. WeatherReal also features a publicly accessible quality control and evaluation framework. This paper details the sources and processing methodologies underlying the dataset, and further illustrates the advantage of in-situ observations in capturing hyper-local and extreme weather through comparative analyses and case studies. Using WeatherReal, we evaluated several data-driven models and compared them with leading numerical models. Our work aims to advance the AI-based weather forecasting research towards a more application-focused and operation-ready approach.
Code (1)
Tasks
Weather ForecastingSimilar Papers 제목 키워드 기반
Verification against in-situ observations for Data-Driven Weather Prediction
Data-driven weather prediction models (DDWPs) have made rapid strides in recent years, demonstrating an ability to approximate Numerical Weather Prediction (NWP) models to a high degree of accuracy. The fast, accurate, a…
From Global to Local: Efficient Regional Weather Downscaling with Global Weather Foundation Model
Accurate regional weather prediction requires resolving fine-scale structure while remaining consistent with global dynamics. Traditional limited area models rely on computationally expensive simulations, while many lear…
MAUSAM: An Observations-focused assessment of Global AI Weather Prediction Models During the South Asian Monsoon
Accurate weather forecasts are critical for societal planning and disaster preparedness. Yet these forecasts remain challenging to produce and evaluate, especially in regions with sparse observational coverage. Current e…
A Benchmark for AI-based Weather Data Assimilation
Recent advancements in Artificial Intelligence (AI) have led to the development of several Large Weather Models (LWMs) that rival State-Of-The-Art (SOTA) Numerical Weather Prediction (NWP) systems. Until now, these model…
Weather ForecastingWXImpactBench: A Disruptive Weather Impact Understanding Benchmark for Evaluating Large Language Models
Climate change adaptation requires the understanding of disruptive weather impacts on society, where large language models (LLMs) might be applicable. However, their effectiveness is under-explored due to the difficulty …
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONQuestion Answering