Time-Series Forecasting, Knowledge Distillation, and Refinement within a Multimodal PDE Foundation Model

2024-09-17 · Derek Jollie, Jingmin Sun, Zecheng Zhang, Hayden Schaeffer

Symbolic encoding has been used in multi-operator learning as a way to embed additional information for distinct time-series data. For spatiotemporal systems described by time-dependent partial differential equations, the equation itself provides an additional modality to identify the system. The utilization of symbolic expressions along side time-series samples allows for the development of multimodal predictive neural networks. A key challenge with current approaches is that the symbolic information, i.e. the equations, must be manually preprocessed (simplified, rearranged, etc.) to match and relate to the existing token library, which increases costs and reduces flexibility, especially when dealing with new differential equations. We propose a new token library based on SymPy to encode differential equations as an additional modality for time-series models. The proposed approach incurs minimal cost, is automated, and maintains high prediction accuracy for forecasting tasks. Additionally, we include a Bayesian filtering module that connects the different modalities to refine the learned equation. This improves the accuracy of the learned symbolic representation and the predicted time-series.

📄 PDF Abstract BibTeX arXiv:2409.11609

Code (1)

jingminsun/prose_v1 공식 구현 jax

Tasks

Knowledge DistillationOperator learningTime SeriesTime Series Forecasting

Methods 이 논문이 사용한 방법론

Library 설명 없음

Distilling Time Series Foundation Models for Efficient Forecasting

2026-01-19 · Yuqi Li, Kuiye Ding, Chuanguang Yang, Szu-Yu Chen 외 arxiv

Time Series foundation models (TSFMs) deliver strong forecasting performance through large-scale pretraining, but their large parameter sizes make deployment costly. While knowledge distillation offers a natural and effe…

Time Series Forecasting Knowledge Distillation Model Compression

xTime: Extreme Event Prediction with Hierarchical Knowledge Distillation and Expert Fusion

2025-10-23 · Quan Li, Wenchao Yu, Suhang Wang, Minhua Lin 외 arxiv

Extreme events frequently occur in real-world time series and often carry significant practical implications. In domains such as climate and healthcare, these events, such as floods, heatwaves, or acute medical episodes,…

Time Series Forecasting Knowledge Distillation

Efficient Multivariate Time Series Forecasting via Calibrated Language Models with Privileged Knowledge Distillation

2025-05-04 · Chenxi Liu, Hao Miao, Qianxiong Xu, Shaowen Zhou 외

Multivariate time series forecasting (MTSF) endeavors to predict future observations given historical data, playing a crucial role in time series data management systems. With advancements in large language models (LLMs)…

Knowledge Distillation Multivariate Time Series Forecasting Time Series Time Series Forecasting

Harmonic Dataset Distillation for Time Series Forecasting

2026-03-04 · Seungha Hong, Sanghwan Jang, Wonbin Kweon, Suyeon Kim 외 arxiv

Time Series forecasting (TSF) in the modern era faces significant computational and storage cost challenges due to the massive scale of real-world data. Dataset Distillation (DD), a paradigm that synthesizes a small, com…

Time Series Forecasting

OccamVTS: Distilling Vision Models to 1% Parameters for Time Series Forecasting

2025-08-03 · Sisuo Lyu, Siru Zhong, Weilin Ruan, Qingxiang Liu 외 arxiv

Time series forecasting is fundamental to diverse applications, with recent approaches leverage large vision models (LVMs) to capture temporal patterns through visual representations. We reveal that while vision models e…