SynopticBench: Evaluating Vision-Language Models on Generating Weather Forecast Discussions of the Future
Recent advances in visual-language models (VLMs) have led to significant improvements in a plethora of complex multimodal tasks like image captioning, report generation, and visual perception. However, generating text from meteorological data is highly challenging because the atmosphere is a chaotic system that is rapidly changing at various spatial and temporal scales. Given the complexity of atmospheric phenomena, it is critical to verifiably quantify the effectiveness of existing VLMs on weather forecasting data. In this work, we present SynopticBench, a high-quality dataset consisting of 1,367,041 text samples of Area Forecast Discussions created by the National Weather Service over the continental United States paired to images of 500mb geopotential height, 2 meter temperature, and 850mb wind velocity in weather forecasts. We also present Synoptic Phenomena Alignment and Coverage Evaluation (SPACE), a novel evaluation framework that can be used to effectively estimate the quality of text descriptions of synoptic weather phenomena. Extensive experiments on generating forecast discussions using state-of-the-art VLMs show the sensitivity of existing evaluation metrics in this domain and enable further exploration into synoptic weather and climate text generation.
Code (0)
등록된 구현이 없습니다.
Tasks
Weather ForecastingImage CaptioningText GenerationSimilar Papers 제목 키워드 기반
Language-driven All-in-one Adverse Weather Removal
All-in-one (AiO) frameworks restore various adverse weather degradations with a single set of networks jointly. To handle various weather conditions, an AiO framework is expected to adaptively learn weather-specific know…
AllDiversityMixture-of-ExpertsGenerating Weather Forecast Texts with Case Based Reasoning
Several techniques have been used to generate weather forecast texts. In this paper, case based reasoning (CBR) is proposed for weather forecast text generation because similar weather conditions occur over time and shou…
RetrievalText GenerationWXImpactBench: A Disruptive Weather Impact Understanding Benchmark for Evaluating Large Language Models
Climate change adaptation requires the understanding of disruptive weather impacts on society, where large language models (LLMs) might be applicable. However, their effectiveness is under-explored due to the difficulty …
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONQuestion AnsweringGenerating Clear Images From Images With Distortions Caused by Adverse Weather Using Generative Adversarial Networks
We presented a method for improving computer vision tasks on images affected by adverse weather conditions, including distortions caused by adherent raindrops. Overcoming the challenge of applying computer vision to imag…
Autonomous DrivingAutonomous VehiclesGenerative Adversarial NetworkImage Reconstruction+1Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models
This paper addresses the limitations of adverse weather image restoration approaches trained on synthetic data when applied to real-world scenarios. We formulate a semi-supervised learning framework employing vision-lang…
Image RestorationLanguage ModelingLanguage ModellingPrompt Learning