paper-with-me

Papers

Transformer Training Strategies for Forecasting Multiple Load Time Series

2023-06-19 · Matthias Hertel, Maximilian Beichter, Benedikt Heidrich, Oliver Neumann, Benjamin Schäfer, Ralf Mikut, Veit Hagenmeyer

In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We evaluate whether a Transformer load forecasting model benefits from a transfer learning strategy, where a global univariate model is trained on the load time series from multiple clients. In experiments with two datasets containing load time series from several hundred clients, we find that the global training strategy is superior to the multivariate and local training strategies used in related work. On average, the global training strategy results in 21.8% and 12.8% lower forecasting errors than the two other strategies, measured across forecasting horizons from one day to one month into the future. A comparison to linear models, multi-layer perceptrons and LSTMs shows that Transformers are effective for load forecasting when they are trained with the global training strategy.

📄 PDF Abstract BibTeX arXiv:2306.10891

Code (1)

kit-iai/transformer-training-strategies 공식 구현 pytorch

Tasks

Load ForecastingTime SeriesTransfer Learning

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Position-Wise Feed-Forward Layer 설명 없음
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Adam 설명 없음
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Similar Papers 제목 키워드 기반

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

2026-07-17 · Matthias Hertel, Sebastian Pütz, Jonathan Kolar, Benjamin Schäfer 외 arxiv

Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for dema…

Hyperparameter Optimization

Transfer Learning in Deep Learning Models for Building Load Forecasting: Case of Limited Data

2023-01-25 · Menna Nawar, Moustafa Shomer, Samy Faddel, Huangjie Gong

Precise load forecasting in buildings could increase the bill savings potential and facilitate optimized strategies for power generation planning. With the rapid evolution of computer science, data-driven techniques, in …

Deep LearningLoad ForecastingTransfer Learning

Initialization Is Critical: Advancing Federated Short-Term Load Forecasting under Load Heterogeneity via Model Initialization

2026-08-28 · Jianing Chen, Vajiheh Farhadi, Yan Li, Thomas La Porta arxiv

Short-term load forecasting (STLF) provides essential information for numerous applications in modern power systems. However, accurate STLF often relies on fine-grained smart-meter data from distributed users, raising in…

Federated Learning

TgDLF2.0: Theory-guided deep-learning for electrical load forecasting via Transformer and transfer learning

2022-10-05 · Jiaxin Gao, WenBo Hu, Dongxiao Zhang, Yuntian Chen

Electrical energy is essential in today's society. Accurate electrical load forecasting is beneficial for better scheduling of electricity generation and saving electrical energy. In this paper, we propose theory-guided …

Deep LearningLoad ForecastingSchedulingTransfer Learning

Differential Evolution Algorithm based Hyper-Parameters Selection of Transformer Neural Network Model for Load Forecasting

2023-07-28 · Anuvab Sen, Arul Rhik Mazumder, Udayon Sen

Accurate load forecasting plays a vital role in numerous sectors, but accurately capturing the complex dynamics of dynamic power systems remains a challenge for traditional statistical models. For these reasons, time-ser…

Load ForecastingTime Series