GOPHER: Categorical probabilistic forecasting withgraph structure via local continuous-time dynamics
We consider the problem of probabilistic forecasting over categories with graph structure, where the dynamics at a vertex depends on its local connectivity structure. We present GOPHER, a method that combines the inductive bias of graph neural networks with neural ODEs to capture the intrinsic local continuous-time dynamics of our probabilistic forecasts. We study the benefits of these two inductive biases by comparing against baseline models that help disentangle the benefits of each. We find that capturing the graph structure is crucial for accurate in-domain probabilistic predictions and more sample efficient models. Surprisingly, our experiments demonstrate that the continuous time evolution inductive bias brings little to no benefit despite reflecting the true probability dynamics.
Code (0)
등록된 구현이 없습니다.
Tasks
Inductive BiasSimilar Papers 제목 키워드 기반
GOPHER: Categorical probabilistic forecasting with graph structure via local continuous-time dynamics
We consider the problem of probabilistic forecasting over categories with graph structure, where the dynamics at a vertex depends on its local connectivity structure. We present GOPHER, a method that combines the inducti…
Inductive BiasWeatherBench Probability: A benchmark dataset for probabilistic medium-range weather forecasting along with deep learning baseline models
WeatherBench is a benchmark dataset for medium-range weather forecasting of geopotential, temperature and precipitation, consisting of preprocessed data, predefined evaluation metrics and a number of baseline models. Wea…
Weather ForecastingLong Short Term Memory Networks for Bandwidth Forecasting in Mobile Broadband Networks under Mobility
Bandwidth forecasting in Mobile Broadband (MBB) networks is a challenging task, particularly when coupled with a degree of mobility. In this work, we introduce HINDSIGHT++, an open-source R-based framework for bandwidth …
AutoMLHyperparameter OptimizationTraining Compute-Optimal Large Language Models
We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget. We find that current large language models are significantly undertrained, a consequence …
AnachronismsAnalogical SimilarityAnalytic EntailmentCausal Judgment+69Probabilistic Hash Embeddings for Online Learning of Categorical Features
We study streaming data with categorical features where the vocabulary of categorical feature values is changing and can even grow unboundedly over time. Feature hashing is commonly used as a pre-processing step to map t…
Recommendation Systems