A Unified Hyperparameter Optimization Pipeline for Transformer-Based Time Series Forecasting Models
Transformer-based models for time series forecasting (TSF) have attracted significant attention in recent years due to their effectiveness and versatility. However, these models often require extensive hyperparameter optimization (HPO) to achieve the best possible performance, and a unified pipeline for HPO in transformer-based TSF remains lacking. In this paper, we present one such pipeline and conduct extensive experiments on several state-of-the-art (SOTA) transformer-based TSF models. These experiments are conducted on standard benchmark datasets to evaluate and compare the performance of different models, generating practical insights and examples. Our pipeline is generalizable beyond transformer-based architectures and can be applied to other SOTA models, such as Mamba and TimeMixer, as demonstrated in our experiments. The goal of this work is to provide valuable guidance to both industry practitioners and academic researchers in efficiently identifying optimal hyperparameters suited to their specific domain applications. The code and complete experimental results are available on GitHub.
Code (1)
Tasks
Hyperparameter OptimizationMambaTime SeriesTime Series ForecastingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FLASH: Fast Bayesian Optimization for Data Analytic Pipelines
Modern data science relies on data analytic pipelines to organize interdependent computational steps. Such analytic pipelines often involve different algorithms across multiple steps, each with its own hyperparameters. T…
Bayesian OptimizationReducing Hyperparameter Tuning Costs in ML, Vision and Language Model Training Pipelines via Memoization-Awareness
The training or fine-tuning of machine learning, vision, and language models is often implemented as a pipeline: a sequence of stages encompassing data preparation, model training and evaluation. In this paper, we exploi…
Bayesian OptimizationGPULanguage ModelingLanguage ModellingBe aware of overfitting by hyperparameter optimization!
Hyperparameter optimization is very frequently employed in machine learning. However, an optimization of a large space of parameters could result in overfitting of models. In recent studies on solubility prediction the a…
Hyperparameter OptimizationRepresentation LearningQuantifying contribution and propagation of error from computational steps, algorithms and hyperparameter choices in image classification pipelines
Data science relies on pipelines that are organized in the form of interdependent computational steps. Each step consists of various candidate algorithms that maybe used for performing a particular function. Each algorit…
Bayesian OptimizationBIG-bench Machine LearningGeneral ClassificationHyperparameter Optimization+2Learning To Exploit the Sequence-Specific Prior Knowledge for Image Processing Pipelines Optimization
The hardware image signal processing (ISP) pipeline is the intermediate layer between the imaging sensor and the downstream application, processing the sensor signal into an RGB image. The ISP is less programmable an…
Hyperparameter OptimizationImage Segmentationobject-detectionObject Detection+1